In [66]:
#insert the photo
from IPython.display import Image
image_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/figs/title_happy.jpg"
Image(filename=image_path, width=600, height=600)
Out[66]:

What is happiness?¶

Happiness, a profound and elusive emotion, is the warm embrace of the soul when life's melodies harmonize with our inner symphony. It's the radiant sunbeam that pierces through the darkest of clouds, illuminating our path with hope and joy. Happiness is not a fleeting moment but a state of being, a tapestry woven from the threads of gratitude, contentment, and love.

It resides in the simple pleasures of life, like the laughter of children, the scent of blooming flowers, or the gentle touch of a loved one's hand. It's the satisfaction derived from pursuing our passions, the exhilaration of reaching new heights, and the serenity found in quiet moments of reflection.

Happiness is a choice, an art of embracing life's imperfections with a heart full of acceptance. It's the empathy we extend to others, the acts of kindness that create ripples of joy, and the connections we forge with fellow travelers on this journey.

In the tapestry of life, happiness is the vibrant thread that weaves through our experiences, making each moment precious. It's the compass that guides us towards a life well-lived, reminding us that true happiness is not found in possessions but in the richness of our human connections and the depths of our souls.

In [ ]:
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

import numpy as np
In [2]:
!pip install seaborn

Requirement already satisfied: seaborn in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (0.12.2)
Requirement already satisfied: numpy!=1.24.0,>=1.17 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from seaborn) (1.24.3)
Requirement already satisfied: pandas>=0.25 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from seaborn) (1.5.3)
Requirement already satisfied: matplotlib!=3.6.1,>=3.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from seaborn) (3.8.2)
Requirement already satisfied: contourpy>=1.0.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (1.0.5)
Requirement already satisfied: cycler>=0.10 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (0.11.0)
Requirement already satisfied: fonttools>=4.22.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (4.25.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (1.4.4)
Requirement already satisfied: packaging>=20.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (23.0)
Requirement already satisfied: pillow>=8 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (9.4.0)
Requirement already satisfied: pyparsing>=2.3.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib!=3.6.1,>=3.1->seaborn) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from pandas>=0.25->seaborn) (2022.7)
Requirement already satisfied: six>=1.5 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from python-dateutil>=2.7->matplotlib!=3.6.1,>=3.1->seaborn) (1.16.0)
In [3]:
cleaned_hm_csv_path = "/Users/chenjianfeng/Desktop/5243/project 1 happy DB/HappyDB-master/happydb/data/cleaned_hm.csv"
demographic_csv_path = "/Users/chenjianfeng/Desktop/5243/project 1 happy DB/HappyDB-master/happydb/data/demographic.csv"

Fortunately, we've got our hands on a dataset focused on happiness and joyous moments. We're all set to embark on an exciting journey of analyzing the Happy Moment Database. Let's dive right in!¶

In [67]:
#insert the photo
from IPython.display import Image
image_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/figs/happy_db.jpg"
Image(filename=image_path, width=600, height=600)
Out[67]:

1: What really matters in our happiness?¶

WordCloud¶

In [26]:
pip install wordcloud

Collecting wordcloud
  Obtaining dependency information for wordcloud from https://files.pythonhosted.org/packages/ac/66/937d1d73389c0b501c928c4d8513653063d2b40272dff70d0e283d8b9144/wordcloud-1.9.3-cp311-cp311-macosx_11_0_arm64.whl.metadata
  Downloading wordcloud-1.9.3-cp311-cp311-macosx_11_0_arm64.whl.metadata (3.4 kB)
Requirement already satisfied: numpy>=1.6.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from wordcloud) (1.24.3)
Requirement already satisfied: pillow in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from wordcloud) (9.4.0)
Requirement already satisfied: matplotlib in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from wordcloud) (3.8.2)
Requirement already satisfied: contourpy>=1.0.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib->wordcloud) (1.0.5)
Requirement already satisfied: cycler>=0.10 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib->wordcloud) (0.11.0)
Requirement already satisfied: fonttools>=4.22.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib->wordcloud) (4.25.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib->wordcloud) (1.4.4)
Requirement already satisfied: packaging>=20.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib->wordcloud) (23.0)
Requirement already satisfied: pyparsing>=2.3.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib->wordcloud) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib->wordcloud) (2.8.2)
Requirement already satisfied: six>=1.5 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from python-dateutil>=2.7->matplotlib->wordcloud) (1.16.0)
Downloading wordcloud-1.9.3-cp311-cp311-macosx_11_0_arm64.whl (168 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 168.3/168.3 kB 966.5 kB/s eta 0:00:00 0:00:01m
Installing collected packages: wordcloud
Successfully installed wordcloud-1.9.3
Note: you may need to restart the kernel to use updated packages.
In [68]:
import pandas as pd
from wordcloud import WordCloud
import matplotlib.pyplot as plt

# Read the CSV file
df = pd.read_csv('/Users/chenjianfeng/Desktop/5243/project 1 happy DB/HappyDB-master/happydb/data/cleaned_hm.csv')

# Concatenate text data from the 'cleaned_hm' column into a long string
text = ' '.join(df['cleaned_hm'].dropna())

# Generate a word cloud
wordcloud = WordCloud(width=800, height=400).generate(text)

# Display the word cloud image
plt.figure(figsize=(10, 7))
plt.imshow(wordcloud, interpolation='bilinear')
plt.axis('off')  # Hide the axis
plt.show()
In [34]:
pip install spacy 

Collecting spacy
  Obtaining dependency information for spacy from https://files.pythonhosted.org/packages/ca/f3/609bb7512cad1f02af13daa23aa433b931da34c502211f29fd47dceff624/spacy-3.7.2-cp311-cp311-macosx_11_0_arm64.whl.metadata
  Downloading spacy-3.7.2-cp311-cp311-macosx_11_0_arm64.whl.metadata (25 kB)
Collecting spacy-legacy<3.1.0,>=3.0.11 (from spacy)
  Downloading spacy_legacy-3.0.12-py2.py3-none-any.whl (29 kB)
Collecting spacy-loggers<2.0.0,>=1.0.0 (from spacy)
  Obtaining dependency information for spacy-loggers<2.0.0,>=1.0.0 from https://files.pythonhosted.org/packages/33/78/d1a1a026ef3af911159398c939b1509d5c36fe524c7b644f34a5146c4e16/spacy_loggers-1.0.5-py3-none-any.whl.metadata
  Downloading spacy_loggers-1.0.5-py3-none-any.whl.metadata (23 kB)
Collecting murmurhash<1.1.0,>=0.28.0 (from spacy)
  Obtaining dependency information for murmurhash<1.1.0,>=0.28.0 from https://files.pythonhosted.org/packages/7a/05/4a3b5c3043c6d84c00bf0f574d326660702b1c10174fe6b44cef3c3dff08/murmurhash-1.0.10-cp311-cp311-macosx_11_0_arm64.whl.metadata
  Downloading murmurhash-1.0.10-cp311-cp311-macosx_11_0_arm64.whl.metadata (2.0 kB)
Collecting cymem<2.1.0,>=2.0.2 (from spacy)
  Obtaining dependency information for cymem<2.1.0,>=2.0.2 from https://files.pythonhosted.org/packages/d7/f6/67babf1439cdd6d46e4e805616bee84981305c80e562320c293712f54034/cymem-2.0.8-cp311-cp311-macosx_11_0_arm64.whl.metadata
  Downloading cymem-2.0.8-cp311-cp311-macosx_11_0_arm64.whl.metadata (8.4 kB)
Collecting preshed<3.1.0,>=3.0.2 (from spacy)
  Obtaining dependency information for preshed<3.1.0,>=3.0.2 from https://files.pythonhosted.org/packages/a8/b3/1a73ba16bab53043fd19dd0a7838ae05c705dccb329404dd4ad5925767f1/preshed-3.0.9-cp311-cp311-macosx_11_0_arm64.whl.metadata
  Downloading preshed-3.0.9-cp311-cp311-macosx_11_0_arm64.whl.metadata (2.2 kB)
Collecting thinc<8.3.0,>=8.1.8 (from spacy)
  Obtaining dependency information for thinc<8.3.0,>=8.1.8 from https://files.pythonhosted.org/packages/c8/43/0ee5796e9bf91eb8e1e032b4ef407e2fbec7cb2e3da9b13f44a2d2f82607/thinc-8.2.2-cp311-cp311-macosx_11_0_arm64.whl.metadata
  Downloading thinc-8.2.2-cp311-cp311-macosx_11_0_arm64.whl.metadata (15 kB)
Collecting wasabi<1.2.0,>=0.9.1 (from spacy)
  Obtaining dependency information for wasabi<1.2.0,>=0.9.1 from https://files.pythonhosted.org/packages/8f/69/26cbf0bad11703241cb84d5324d868097f7a8faf2f1888354dac8883f3fc/wasabi-1.1.2-py3-none-any.whl.metadata
  Downloading wasabi-1.1.2-py3-none-any.whl.metadata (28 kB)
Collecting srsly<3.0.0,>=2.4.3 (from spacy)
  Obtaining dependency information for srsly<3.0.0,>=2.4.3 from https://files.pythonhosted.org/packages/1b/d7/0800af1a75008b3a6a6a24f3efd165f2d2208076e9b8a4b11b66f16217f3/srsly-2.4.8-cp311-cp311-macosx_11_0_arm64.whl.metadata
  Downloading srsly-2.4.8-cp311-cp311-macosx_11_0_arm64.whl.metadata (20 kB)
Collecting catalogue<2.1.0,>=2.0.6 (from spacy)
  Obtaining dependency information for catalogue<2.1.0,>=2.0.6 from https://files.pythonhosted.org/packages/9e/96/d32b941a501ab566a16358d68b6eb4e4acc373fab3c3c4d7d9e649f7b4bb/catalogue-2.0.10-py3-none-any.whl.metadata
  Downloading catalogue-2.0.10-py3-none-any.whl.metadata (14 kB)
Collecting weasel<0.4.0,>=0.1.0 (from spacy)
  Obtaining dependency information for weasel<0.4.0,>=0.1.0 from https://files.pythonhosted.org/packages/d5/e5/b63b8e255d89ba4155972990d42523251d4d1368c4906c646597f63870e2/weasel-0.3.4-py3-none-any.whl.metadata
  Downloading weasel-0.3.4-py3-none-any.whl.metadata (4.7 kB)
Collecting typer<0.10.0,>=0.3.0 (from spacy)
  Downloading typer-0.9.0-py3-none-any.whl (45 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 45.9/45.9 kB 2.6 MB/s eta 0:00:00
Requirement already satisfied: smart-open<7.0.0,>=5.2.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (5.2.1)
Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (4.66.1)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (2.31.0)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (1.10.12)
Requirement already satisfied: jinja2 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (3.1.2)
Requirement already satisfied: setuptools in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (68.0.0)
Requirement already satisfied: packaging>=20.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (23.0)
Collecting langcodes<4.0.0,>=3.2.0 (from spacy)
  Downloading langcodes-3.3.0-py3-none-any.whl (181 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.6/181.6 kB 3.3 MB/s eta 0:00:00a 0:00:01
Requirement already satisfied: numpy>=1.19.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (1.24.3)
Requirement already satisfied: typing-extensions>=4.2.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy) (4.7.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy) (2023.7.22)
Collecting blis<0.8.0,>=0.7.8 (from thinc<8.3.0,>=8.1.8->spacy)
  Obtaining dependency information for blis<0.8.0,>=0.7.8 from https://files.pythonhosted.org/packages/a8/73/0a9d4e7f6e78ef270e3a4532b17e060a02087590cf615ba9943fd1a283e9/blis-0.7.11-cp311-cp311-macosx_11_0_arm64.whl.metadata
  Downloading blis-0.7.11-cp311-cp311-macosx_11_0_arm64.whl.metadata (7.4 kB)
Collecting confection<1.0.0,>=0.0.1 (from thinc<8.3.0,>=8.1.8->spacy)
  Obtaining dependency information for confection<1.0.0,>=0.0.1 from https://files.pythonhosted.org/packages/39/78/f9d18da7b979a2e6007bfcea2f3c8cc02ed210538ae1ce7e69092aed7b18/confection-0.1.4-py3-none-any.whl.metadata
  Downloading confection-0.1.4-py3-none-any.whl.metadata (19 kB)
Requirement already satisfied: click<9.0.0,>=7.1.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from typer<0.10.0,>=0.3.0->spacy) (8.0.4)
Collecting cloudpathlib<0.17.0,>=0.7.0 (from weasel<0.4.0,>=0.1.0->spacy)
  Obtaining dependency information for cloudpathlib<0.17.0,>=0.7.0 from https://files.pythonhosted.org/packages/0f/6e/45b57a7d4573d85d0b0a39d99673dc1f5eea9d92a1a4603b35e968fbf89a/cloudpathlib-0.16.0-py3-none-any.whl.metadata
  Downloading cloudpathlib-0.16.0-py3-none-any.whl.metadata (14 kB)
Requirement already satisfied: MarkupSafe>=2.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from jinja2->spacy) (2.1.1)
Downloading spacy-3.7.2-cp311-cp311-macosx_11_0_arm64.whl (6.5 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.5/6.5 MB 27.1 MB/s eta 0:00:0000:0100:01
Downloading catalogue-2.0.10-py3-none-any.whl (17 kB)
Downloading cymem-2.0.8-cp311-cp311-macosx_11_0_arm64.whl (41 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.2/41.2 kB 8.3 MB/s eta 0:00:00
Downloading murmurhash-1.0.10-cp311-cp311-macosx_11_0_arm64.whl (26 kB)
Downloading preshed-3.0.9-cp311-cp311-macosx_11_0_arm64.whl (128 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 128.8/128.8 kB 28.6 MB/s eta 0:00:00
Downloading spacy_loggers-1.0.5-py3-none-any.whl (22 kB)
Downloading srsly-2.4.8-cp311-cp311-macosx_11_0_arm64.whl (488 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 488.4/488.4 kB 39.8 MB/s eta 0:00:00
Downloading thinc-8.2.2-cp311-cp311-macosx_11_0_arm64.whl (778 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 778.9/778.9 kB 36.2 MB/s eta 0:00:00
Downloading wasabi-1.1.2-py3-none-any.whl (27 kB)
Downloading weasel-0.3.4-py3-none-any.whl (50 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.1/50.1 kB 7.3 MB/s eta 0:00:00
Downloading blis-0.7.11-cp311-cp311-macosx_11_0_arm64.whl (1.1 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 44.6 MB/s eta 0:00:00
Downloading cloudpathlib-0.16.0-py3-none-any.whl (45 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 45.0/45.0 kB 7.4 MB/s eta 0:00:00
Downloading confection-0.1.4-py3-none-any.whl (35 kB)
Installing collected packages: cymem, wasabi, typer, spacy-loggers, spacy-legacy, murmurhash, langcodes, cloudpathlib, catalogue, blis, srsly, preshed, confection, weasel, thinc, spacy
Successfully installed blis-0.7.11 catalogue-2.0.10 cloudpathlib-0.16.0 confection-0.1.4 cymem-2.0.8 langcodes-3.3.0 murmurhash-1.0.10 preshed-3.0.9 spacy-3.7.2 spacy-legacy-3.0.12 spacy-loggers-1.0.5 srsly-2.4.8 thinc-8.2.2 typer-0.9.0 wasabi-1.1.2 weasel-0.3.4
Note: you may need to restart the kernel to use updated packages.
In [38]:
!pip install spacy wordcloud matplotlib
!python -m spacy download en_core_web_sm

Requirement already satisfied: spacy in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (3.7.2)
Requirement already satisfied: wordcloud in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (1.9.3)
Requirement already satisfied: matplotlib in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (3.8.2)
Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (3.0.12)
Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (1.0.5)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (1.0.10)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (2.0.8)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (3.0.9)
Requirement already satisfied: thinc<8.3.0,>=8.1.8 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (8.2.2)
Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (1.1.2)
Requirement already satisfied: srsly<3.0.0,>=2.4.3 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (2.4.8)
Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (2.0.10)
Requirement already satisfied: weasel<0.4.0,>=0.1.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (0.3.4)
Requirement already satisfied: typer<0.10.0,>=0.3.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (0.9.0)
Requirement already satisfied: smart-open<7.0.0,>=5.2.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (5.2.1)
Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (4.66.1)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (2.31.0)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (1.10.12)
Requirement already satisfied: jinja2 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (3.1.2)
Requirement already satisfied: setuptools in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (68.0.0)
Requirement already satisfied: packaging>=20.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (23.0)
Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (3.3.0)
Requirement already satisfied: numpy>=1.19.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy) (1.24.3)
Requirement already satisfied: pillow in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from wordcloud) (9.4.0)
Requirement already satisfied: contourpy>=1.0.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib) (1.0.5)
Requirement already satisfied: cycler>=0.10 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib) (0.11.0)
Requirement already satisfied: fonttools>=4.22.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib) (4.25.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib) (1.4.4)
Requirement already satisfied: pyparsing>=2.3.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from matplotlib) (2.8.2)
Requirement already satisfied: typing-extensions>=4.2.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy) (4.7.1)
Requirement already satisfied: six>=1.5 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from python-dateutil>=2.7->matplotlib) (1.16.0)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy) (2023.7.22)
Requirement already satisfied: blis<0.8.0,>=0.7.8 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from thinc<8.3.0,>=8.1.8->spacy) (0.7.11)
Requirement already satisfied: confection<1.0.0,>=0.0.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from thinc<8.3.0,>=8.1.8->spacy) (0.1.4)
Requirement already satisfied: click<9.0.0,>=7.1.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from typer<0.10.0,>=0.3.0->spacy) (8.0.4)
Requirement already satisfied: cloudpathlib<0.17.0,>=0.7.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from weasel<0.4.0,>=0.1.0->spacy) (0.16.0)
Requirement already satisfied: MarkupSafe>=2.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from jinja2->spacy) (2.1.1)
Collecting en-core-web-sm==3.7.1
  Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.7.1/en_core_web_sm-3.7.1-py3-none-any.whl (12.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.8/12.8 MB 46.6 MB/s eta 0:00:00a 0:00:01
Requirement already satisfied: spacy<3.8.0,>=3.7.2 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from en-core-web-sm==3.7.1) (3.7.2)
Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (3.0.12)
Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (1.0.5)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (1.0.10)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2.0.8)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (3.0.9)
Requirement already satisfied: thinc<8.3.0,>=8.1.8 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (8.2.2)
Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (1.1.2)
Requirement already satisfied: srsly<3.0.0,>=2.4.3 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2.4.8)
Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2.0.10)
Requirement already satisfied: weasel<0.4.0,>=0.1.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (0.3.4)
Requirement already satisfied: typer<0.10.0,>=0.3.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (0.9.0)
Requirement already satisfied: smart-open<7.0.0,>=5.2.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (5.2.1)
Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (4.66.1)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2.31.0)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (1.10.12)
Requirement already satisfied: jinja2 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (3.1.2)
Requirement already satisfied: setuptools in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (68.0.0)
Requirement already satisfied: packaging>=20.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (23.0)
Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (3.3.0)
Requirement already satisfied: numpy>=1.19.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (1.24.3)
Requirement already satisfied: typing-extensions>=4.2.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (4.7.1)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2.0.4)
Requirement already satisfied: idna<4,>=2.5 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from requests<3.0.0,>=2.13.0->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2023.7.22)
Requirement already satisfied: blis<0.8.0,>=0.7.8 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from thinc<8.3.0,>=8.1.8->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (0.7.11)
Requirement already satisfied: confection<1.0.0,>=0.0.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from thinc<8.3.0,>=8.1.8->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (0.1.4)
Requirement already satisfied: click<9.0.0,>=7.1.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from typer<0.10.0,>=0.3.0->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (8.0.4)
Requirement already satisfied: cloudpathlib<0.17.0,>=0.7.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from weasel<0.4.0,>=0.1.0->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (0.16.0)
Requirement already satisfied: MarkupSafe>=2.0 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from jinja2->spacy<3.8.0,>=3.7.2->en-core-web-sm==3.7.1) (2.1.1)
Installing collected packages: en-core-web-sm
Successfully installed en-core-web-sm-3.7.1
✔ Download and installation successful
You can now load the package via spacy.load('en_core_web_sm')
In [41]:
import spacy

# Load the spaCy model
nlp = spacy.load("en_core_web_sm")

# Read the data
df = pd.read_csv('/Users/chenjianfeng/Desktop/5243/project 1 happy DB/HappyDB-master/happydb/data/cleaned_hm.csv')

# Concatenate all the text data
text = ' '.join(df['cleaned_hm'].dropna())

# Define a function to process large text
def process_large_text(text, chunk_size=1000000):
    """
    Split a large text into smaller chunks and process each chunk using spaCy.
    """
    nlp.max_length = chunk_size
    doc_chunks = []
    for i in range(0, len(text), chunk_size):
        doc_chunk = nlp(text[i:i+chunk_size])
        doc_chunks.append(doc_chunk)
    return doc_chunks

# Call the function to process the text
doc_chunks = process_large_text(text)

# Now I can perform further processing on each document chunk, such as extracting information, generating word clouds, etc.
In [42]:
from wordcloud import WordCloud
import matplotlib.pyplot as plt

# Merge all the words from the document chunks
all_words = []
for doc_chunk in doc_chunks:
    all_words.extend([token.text for token in doc_chunk])

# Concatenate the processed words into a long string
cleaned_text = ' '.join(all_words)

# Generate a word cloud using WordCloud, you can specify additional parameters like stopwords
wordcloud = WordCloud(
    width=800,
    height=400,
    background_color='white'
).generate(cleaned_text)

# Display the generated word cloud
plt.figure(figsize=(15, 7))
plt.imshow(wordcloud, interpolation='bilinear')
plt.axis('off')
plt.show()
In [44]:
from nltk.sentiment.vader import SentimentIntensityAnalyzer
from wordcloud import WordCloud
import matplotlib.pyplot as plt

# Initialize the sentiment analyzer
sia = SentimentIntensityAnalyzer()

# Read the data
df = pd.read_csv('/Users/chenjianfeng/Desktop/5243/project 1 happy DB/HappyDB-master/happydb/data/cleaned_hm.csv')

# Concatenate all the text data
text = ' '.join(df['cleaned_hm'].dropna())

# Perform sentiment analysis on the text and accumulate sentiment scores
word_sentiments = {}  # Dictionary for word sentiments
for word in text.split():
    sentiment = sia.polarity_scores(word)['compound']
    word_sentiments[word] = word_sentiments.get(word, 0) + sentiment

# Generate a word cloud based on accumulated sentiment scores for words
wordcloud = WordCloud(
    width=800,
    height=400,
    background_color='white'
).generate_from_frequencies(word_sentiments)

# Display the generated word cloud
plt.figure(figsize=(15, 7))
plt.imshow(wordcloud, interpolation='bilinear')
plt.axis('off')
plt.show()
In [50]:
pip install textblob

Collecting textblob
  Downloading textblob-0.17.1-py2.py3-none-any.whl (636 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 636.8/636.8 kB 2.6 MB/s eta 0:00:00a 0:00:01
Requirement already satisfied: nltk>=3.1 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from textblob) (3.8.1)
Requirement already satisfied: click in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from nltk>=3.1->textblob) (8.0.4)
Requirement already satisfied: joblib in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from nltk>=3.1->textblob) (1.2.0)
Requirement already satisfied: regex>=2021.8.3 in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from nltk>=3.1->textblob) (2022.7.9)
Requirement already satisfied: tqdm in /Users/chenjianfeng/anaconda3/lib/python3.11/site-packages (from nltk>=3.1->textblob) (4.66.1)
Installing collected packages: textblob
Successfully installed textblob-0.17.1
Note: you may need to restart the kernel to use updated packages.
In [52]:
from textblob import TextBlob
from wordcloud import WordCloud
import matplotlib.pyplot as plt
import nltk
from nltk.corpus import stopwords

# Download NLTK stopwords
nltk.download('stopwords')

# Read the data
df = pd.read_csv('/Users/chenjianfeng/Desktop/5243/project 1 happy DB/HappyDB-master/happydb/data/cleaned_hm.csv')

# Concatenate all the text data
text = ' '.join(df['cleaned_hm'].dropna())

# Perform sentiment analysis using TextBlob
blob = TextBlob(text)

# Get the sentiment score of the text
sentiment_score = blob.sentiment.polarity

# Tokenize the text
words = blob.words

# Remove stopwords
stop_words = set(stopwords.words('english'))
filtered_words = [word for word in words if word.lower() not in stop_words]

# Create a dictionary of word sentiments
word_sentiments = {}
for word in filtered_words:
    word_sentiments[word] = blob.word_counts[word] * sentiment_score

# Generate a word cloud based on accumulated sentiment scores for words
wordcloud = WordCloud(
    width=800,
    height=400,
    background_color='white'
).generate_from_frequencies(word_sentiments)

# Display the generated word cloud
plt.figure(figsize=(15, 7))
plt.imshow(wordcloud, interpolation='bilinear')
plt.axis('off')
plt.show()

[nltk_data] Downloading package stopwords to
[nltk_data]     /Users/chenjianfeng/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!

📒 In the HappyDB dataset, regardless of whether stop words are removed, we can distinctly see a few high-frequency words in the cumulative sentiment scores of the language model: work, friend, family, time, birthday, school, and more. This demonstrates that moments of happiness are ever-present in our lives. My parents often tell me that marriage is a significant lifelong investment that greatly influences one's happiness in the latter part of life, and I'm very much looking forward to finding a partner with whom I can grow together. Hence, my theme today revolves around the relationship between marriage and happiness. Follow in my footsteps! 👣

Here's a more idiomatic English version:

Within the HappyDB dataset, whether or not we filter out stop words, the sentiment scores from the language model clearly highlight several prevalent words: work, friend, family, time, birthday, school, among others. This underscores that moments of happiness are intertwined with our daily existence. My parents often emphasize that marriage is a profound lifelong commitment that significantly shapes our happiness in life's second half, and I eagerly anticipate finding a companion to grow alongside. Therefore, today's topic is the intricate connection between marriage and happiness. Let's embark on this journey together! 👣

2. Does getting married guarantee happiness for us? What about the single nobility?¶

In [69]:
#insert the photo
from IPython.display import Image
image_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/figs/marital_happy.jpg"
Image(filename=image_path, width=600, height=600)
Out[69]:

Analyze which group has a stronger sense of happiness, married or single individuals.¶

read clean_hm.csv and degographic.csv file

In [4]:
clean_hm_df = pd.read_csv(cleaned_hm_csv_path)
clean_hm_df.head()
Out[4]:
hmid wid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category
0 27673 2053 24h I went on a successful date with someone I fel... I went on a successful date with someone I fel... True 1 NaN affection
1 27674 2 24h I was happy when my son got 90% marks in his e... I was happy when my son got 90% marks in his e... True 1 NaN affection
2 27675 1936 24h I went to the gym this morning and did yoga. I went to the gym this morning and did yoga. True 1 NaN exercise
3 27676 206 24h We had a serious talk with some friends of our... We had a serious talk with some friends of our... True 2 bonding bonding
4 27677 6227 24h I went with grandchildren to butterfly display... I went with grandchildren to butterfly display... True 1 NaN affection
In [5]:
demographic_df = pd.read_csv(demographic_csv_path)
demographic_df.head()
Out[5]:
wid age country gender marital parenthood
0 1 37.0 USA m married y
1 2 29.0 IND m married y
2 3 25 IND m single n
3 4 32 USA m married y
4 5 29 USA m married y
In [6]:
clean_hm_df.dropna(subset=['ground_truth_category'],inplace=True)
In [7]:
merge_df = demographic_df.merge(clean_hm_df,on='wid')
In [8]:
merge_df.head()
Out[8]:
wid age country gender marital parenthood hmid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category
0 1 37.0 USA m married y 28054 24h My mother called out of the blue to tell me ho... My mother called out of the blue to tell me ho... True 1 affection affection
1 1 37.0 USA m married y 57637 24h I got to take the day off from my part time jo... I got to take the day off from my part time jo... True 1 affection affection
2 1 37.0 USA m married y 64498 3m my son had a great time on his 8th birthday. my son had a great time on his 8th birthday. True 1 affection affection
3 1 37.0 USA m married y 80306 3m I got to have a three day weekend which i spen... I got to have a three day weekend which i spen... True 1 affection affection
4 1 37.0 USA m married y 88307 3m My fiance proposed to me in front of my family... My fiance proposed to me in front of my family... True 1 affection affection
In [9]:
merge_df = merge_df[merge_df['country'] == 'USA']
In [10]:
merge_df['age'] = pd.to_numeric(merge_df['age'], errors='coerce')
merge_df['age'] = merge_df['age'].fillna(0).astype(int)
adult_df = merge_df[merge_df['age']>16]
In [11]:
adult_df.head()
Out[11]:
wid age country gender marital parenthood hmid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category
0 1 37 USA m married y 28054 24h My mother called out of the blue to tell me ho... My mother called out of the blue to tell me ho... True 1 affection affection
1 1 37 USA m married y 57637 24h I got to take the day off from my part time jo... I got to take the day off from my part time jo... True 1 affection affection
2 1 37 USA m married y 64498 3m my son had a great time on his 8th birthday. my son had a great time on his 8th birthday. True 1 affection affection
3 1 37 USA m married y 80306 3m I got to have a three day weekend which i spen... I got to have a three day weekend which i spen... True 1 affection affection
4 1 37 USA m married y 88307 3m My fiance proposed to me in front of my family... My fiance proposed to me in front of my family... True 1 affection affection
In [12]:
adult_wid_unique_df = adult_df.drop_duplicates(subset=['wid'])

Surveys in the United States indicate that single individuals report a stronger sense of happiness.¶

In [70]:
#insert the photo
from IPython.display import Image
image_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/figs/single_happy.jpg"
Image(filename=image_path, width=600, height=600)
Out[70]:
In [13]:
marital_series = adult_wid_unique_df['marital'].value_counts()
sns.set_theme(style="whitegrid", palette="pastel")
barplot  = sns.barplot(x=marital_series.index,y=marital_series.values)
for p in barplot.patches:
    barplot.annotate(format(p.get_height(), '.1f'), 
                     (p.get_x() + p.get_width() / 2., p.get_height()),
                     ha = 'center', va = 'center', 
                     xytext = (0, 9), 
                     textcoords = 'offset points')

📒 The bar chart illustrates the number of individuals who experience happiness in various marital statuses. The data reveal that single individuals in the United States report a strong sense of happiness, with a proportion significantly higher than that of other groups. It is evident that the single population in America enjoys a level of happiness and joy that surpasses expectations.

Here's a more idiomatic English rewrite:

The bar graph presents the count of people experiencing joy across different marital statuses. It's clear from the data that those who are single in the U.S. have a notably robust sense of happiness, standing out with a significantly higher ratio than other demographics. This discovery highlights that the lives of American singles are filled with more happiness and joy than one might presume.

In [14]:
bins = [16, 20, 25, 30,35,40,45,50,55,60]
labels = ['16-20', '21-25', '26-30','31-35', '36-40', '41-45','46-50', '51-55', '56-60']
adult_wid_unique_df['range'] = pd.cut(adult_wid_unique_df['age'], bins=bins, labels=labels)

/var/folders/x1/cmq6ht1x7ygffd5zpgqj2kv80000gn/T/ipykernel_68554/3292791220.py:3: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  adult_wid_unique_df['range'] = pd.cut(adult_wid_unique_df['age'], bins=bins, labels=labels)
In [15]:
range_count_series = adult_wid_unique_df['range'].value_counts()
barplot  = sns.barplot(x=range_count_series.index,y=range_count_series.values)

for p in barplot.patches:
    barplot.annotate(format(p.get_height(), '.1f'), 
                     (p.get_x() + p.get_width() / 2., p.get_height()),
                     ha = 'center', va = 'center', 
                     xytext = (0, 9), 
                     textcoords = 'offset points')

The chart above segments data across age groups in increments of five years, from 16 to 60, creating nine distinct age bands. This stratification allows us to discern the age distribution of the survey respondents, revealing that a significant majority are concentrated in the 21 to 35 age range. This distribution could be indicative of the demographic makeup of the broader society.

Brainstorimg 🌟🌟🌟 : Supplement: You can dig deeper into this area, analyze the distribution of different proportions of men and women, and obtain the number of happy sentences for each age group to investigate the differences between men and women's happiness.

In [16]:
married_range_count = adult_wid_unique_df[adult_wid_unique_df['marital']=='married']['range'].value_counts()

married_range_count_dict = married_range_count.to_dict()

married_range_count_dict
Out[16]:
{'31-35': 456,
 '26-30': 435,
 '36-40': 309,
 '41-45': 172,
 '21-25': 132,
 '51-55': 96,
 '46-50': 93,
 '56-60': 62,
 '16-20': 5}
In [17]:
married_range_count_updated_dict = {}
def update_list_to_cumulative_sum(lst):
    for i in range(1, len(lst)):
        lst[i] += lst[i - 1]
    return lst

updated_list = update_list_to_cumulative_sum(list(married_range_count_dict.values()))

for idx,key in  enumerate(married_range_count_dict.keys()):
    married_range_count_updated_dict[key] = updated_list[idx]

married_range_count_series = pd.Series(married_range_count_updated_dict)
married_range_count_updated_dict
Out[17]:
{'31-35': 456,
 '26-30': 891,
 '36-40': 1200,
 '41-45': 1372,
 '21-25': 1504,
 '51-55': 1600,
 '46-50': 1693,
 '56-60': 1755,
 '16-20': 1760}
In [18]:
g = sns.relplot(x=married_range_count_series.index,y=married_range_count_series.values,kind='scatter')
ax = g.axes[0,0]
ax.plot(married_range_count_series.index, married_range_count_series.values, color='gray', linestyle='--')
barplot  = sns.barplot(x=married_range_count.index,y=married_range_count.values,ax=ax)

for p in barplot.patches:
    barplot.annotate(format(p.get_height(), '.1f'), 
                     (p.get_x() + p.get_width() / 2., p.get_height()),
                     ha = 'center', va = 'center', 
                     xytext = (0, 9), 
                     textcoords = 'offset points')

📒From the perspective of the distribution of single people in various age ranges, first let's look at the histogram below, and combined with the distribution chart of single age ranges below, it is not difficult to see that the proportion of people aged 31-40 is larger than that of single people, which may be reflected from one side, and then After enjoying the happiness of single aristocrats, people in their 30s gradually began to consider marriage and family. Therefore, in the 30-40 range of the chart, the proportion of people who enjoy the happiness brought by marriage is larger than that of those who enjoy the happiness of being single. The above line chart also reflects from the side that people gradually consider marriage after the age of 20.

🌟🌟🌟Note: I just thought about this part and it still needs to be discussed. Later, I will further analyze the base ratio. It is okay not to explain the line chart. For the time being, it does not necessarily represent the growth trend of marriage.

In [19]:
sinle_range_count = adult_wid_unique_df[adult_wid_unique_df['marital']=='single']['range'].value_counts()
barplot  = sns.barplot(x=sinle_range_count.index,y=sinle_range_count.values)

for p in barplot.patches:
    barplot.annotate(format(p.get_height(), '.1f'), 
                     (p.get_x() + p.get_width() / 2., p.get_height()),
                     ha = 'center', va = 'center', 
                     xytext = (0, 9), 
                     textcoords = 'offset points')

📒Judging from the distribution of single people in various age ranges, it may reflect that in the 21-30 age range, young people are more enjoying the happiness brought by being single, and have not yet considered getting married, forming a family, or having children. Parenting.

In [20]:
adult_df.head()
Out[20]:
wid age country gender marital parenthood hmid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category
0 1 37 USA m married y 28054 24h My mother called out of the blue to tell me ho... My mother called out of the blue to tell me ho... True 1 affection affection
1 1 37 USA m married y 57637 24h I got to take the day off from my part time jo... I got to take the day off from my part time jo... True 1 affection affection
2 1 37 USA m married y 64498 3m my son had a great time on his 8th birthday. my son had a great time on his 8th birthday. True 1 affection affection
3 1 37 USA m married y 80306 3m I got to have a three day weekend which i spen... I got to have a three day weekend which i spen... True 1 affection affection
4 1 37 USA m married y 88307 3m My fiance proposed to me in front of my family... My fiance proposed to me in front of my family... True 1 affection affection

The sense of happiness among single individuals is often reflected in the joy of achieving continuous progress in work and studies and gaining recognition from others, which embodies their sense of life's value.¶

In [71]:
#insert the photo
from IPython.display import Image
image_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/figs/career_happy.jpg"
Image(filename=image_path, width=600, height=600)
Out[71]:
In [21]:
single_series = adult_df[adult_df['marital']=='single']['predicted_category'].value_counts()
barplot  = sns.barplot(x=single_series.index,y=single_series.values)
barplot.set_xticklabels(barplot.get_xticklabels(), rotation=90)
for p in barplot.patches:
    barplot.annotate(format(p.get_height(), '.1f'), 
                     (p.get_x() + p.get_width() / 2., p.get_height()),
                     ha = 'center', va = 'center', 
                     xytext = (0, 9), 
                     textcoords = 'offset points')

📒This is a distribution map of different happiness types of single people. Comparing the figure below, it is easy to see that single young people are better than married people in terms of exercise, leisure, enjoy_the_moment, bonding, etc. Single people are far better than married people of the same age. They are more keen on various social activities, such as dining out, exercising, taking elective art and music courses, participating in various public and charity activities, etc. In particular, the achievement item is higher than that of married people, which shows that they still focus on career and value career and academic achievements more.

It is the fondness for the caring atmosphere of family that leads them to happily walk down the aisle into matrimony.¶

In [72]:
#insert the photo
from IPython.display import Image
image_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/figs/parenthood_happy.jpg"
Image(filename=image_path, width=600, height=600)
Out[72]:
In [22]:
married_series = adult_df[adult_df['marital']=='married']['predicted_category'].value_counts()
barplot  = sns.barplot(x=married_series.index,y=married_series.values)
barplot.set_xticklabels(barplot.get_xticklabels(), rotation=90)
for p in barplot.patches:
    barplot.annotate(format(p.get_height(), '.1f'), 
                     (p.get_x() + p.get_width() / 2., p.get_height()),
                     ha = 'center', va = 'center', 
                     xytext = (0, 9), 
                     textcoords = 'offset points')

📒This is a distribution map of different happiness types of single people. Comparing the figure below, it is easy to see that single young people are better than married people in terms of exercise, leisure, enjoy_the_moment, bonding, etc. Single people are far better than married people of the same age. They are more keen on various social activities, such as dining out, exercising, taking elective art and music courses, participating in various public and charity activities, etc. In particular, the achievement item is higher than that of married people, which shows that they still focus on career and value career and academic achievements more.

In marriage, the source of happiness often stems from the affection shared between partners.¶

In [73]:
#insert the photo
from IPython.display import Image
image_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/figs/partner_happy.jpg"
Image(filename=image_path, width=600, height=600)
Out[73]:
In [23]:
married_parenthood_series = adult_df[adult_df['marital']=='married']['parenthood'].value_counts()
barplot  = sns.barplot(x=married_parenthood_series.index,y=married_parenthood_series.values)
for p in barplot.patches:
    barplot.annotate(format(p.get_height(), '.1f'), 
                     (p.get_x() + p.get_width() / 2., p.get_height()),
                     ha = 'center', va = 'center', 
                     xytext = (0, 9), 
                     textcoords = 'offset points')

📒Among married people, comparing those with children and those without children, it is not difficult to see that the proportion of married people with children is larger. Maybe the source of happiness for married people is affection, and affection may be spending time with family members. Sharing happy times together may make people with children feel more happy. This may also be one of the reasons for forming a family and getting married.

Explore: Result comparison in terms of happy moment statistcis and sentiment analysis model¶

In [54]:
df = pd.read_csv("sementic_cleaned_hm.csv")
df.head()
In [ ]:
demographic_csv_path = "/Users/chenjianfeng/Desktop/5243/project 1 happy DB/HappyDB-master/happydb/data/demographic.csv"
demographic_df = pd.read_csv(demographic_csv_path)
demographic_df.head()
Out[ ]:
wid age country gender marital parenthood
0 1 37.0 USA m married y
1 2 29.0 IND m married y
2 3 25 IND m single n
3 4 32 USA m married y
4 5 29 USA m married y
In [ ]:
sns.set_theme(style="whitegrid", palette="pastel")
ax = sns.boxplot(x='predicted_category', y='positive', data=df)
ax.set_xticklabels(ax.get_xticklabels(),rotation=30)

C:\Users\yhyxx\AppData\Local\Temp\ipykernel_6836\268421861.py:3: UserWarning: FixedFormatter should only be used together with FixedLocator
  ax.set_xticklabels(ax.get_xticklabels(),rotation=30)
Out[ ]:
[Text(0, 0, 'affection'),
 Text(1, 0, 'exercise'),
 Text(2, 0, 'bonding'),
 Text(3, 0, 'leisure'),
 Text(4, 0, 'achievement'),
 Text(5, 0, 'enjoy_the_moment'),
 Text(6, 0, 'nature')]
In [ ]:
df_ground_truth = df.dropna(subset=['ground_truth_category'])
In [ ]:
ax = sns.boxplot(x='predicted_category', y='positive', data=df_ground_truth)
ax.set_xticklabels(ax.get_xticklabels(),rotation=30)

C:\Users\yhyxx\AppData\Local\Temp\ipykernel_6836\2889382971.py:2: UserWarning: FixedFormatter should only be used together with FixedLocator
  ax.set_xticklabels(ax.get_xticklabels(),rotation=30)
Out[ ]:
[Text(0, 0, 'bonding'),
 Text(1, 0, 'leisure'),
 Text(2, 0, 'affection'),
 Text(3, 0, 'enjoy_the_moment'),
 Text(4, 0, 'achievement'),
 Text(5, 0, 'nature'),
 Text(6, 0, 'exercise')]
In [ ]:
ax = sns.boxplot(x='predicted_category', y='negative', data=df_ground_truth)
ax.set_xticklabels(ax.get_xticklabels(),rotation=30)

C:\Users\yhyxx\AppData\Local\Temp\ipykernel_6836\1198910182.py:2: UserWarning: FixedFormatter should only be used together with FixedLocator
  ax.set_xticklabels(ax.get_xticklabels(),rotation=30)
Out[ ]:
[Text(0, 0, 'bonding'),
 Text(1, 0, 'leisure'),
 Text(2, 0, 'affection'),
 Text(3, 0, 'enjoy_the_moment'),
 Text(4, 0, 'achievement'),
 Text(5, 0, 'nature'),
 Text(6, 0, 'exercise')]
In [ ]:
merge_df = df_ground_truth.merge(demographic_df,on='wid')
merge_df.head()
Out[ ]:
Unnamed: 0 hmid wid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category positive negative neutral age country gender marital parenthood
0 3 27676 206 24h We had a serious talk with some friends of our... We had a serious talk with some friends of our... True 2 bonding bonding 0.954003 0.044106 0.001891 28 DNK f married n
1 21745 49534 206 24h I bought my wedding shoes! I bought my wedding shoes! True 1 achievement achievement 0.221079 0.271567 0.507355 28 DNK f married n
2 24645 52451 206 24h My new watch arrived in the mail! My new watch arrived in the mail! True 1 achievement achievement 0.146610 0.213970 0.639420 28 DNK f married n
3 56163 84125 206 3m We watched Rocky Horror Picture Show for the f... We watched Rocky Horror Picture Show for the f... True 1 leisure leisure 0.717492 0.189807 0.092702 28 DNK f married n
4 57156 85125 206 3m We played Mario Kart while drunk. We played Mario Kart while drunk. True 1 leisure leisure 0.649013 0.241162 0.109825 28 DNK f married n
In [ ]:
merge_df['age'] = pd.to_numeric(merge_df['age'], errors='coerce')

bins = [16, 20, 25, 30,35,40,45,50,55,60]
labels = ['16-20', '21-25', '26-30','31-35', '36-40', '41-45','46-50', '51-55', '56-60']
merge_df['range'] = pd.cut(merge_df['age'], bins=bins, labels=labels)
In [ ]:
df_age_category_df = merge_df[['ground_truth_category','range','positive']]
df_age_category_df.head()
Out[ ]:
ground_truth_category range positive
0 bonding 26-30 0.954003
1 achievement 26-30 0.221079
2 achievement 26-30 0.146610
3 leisure 26-30 0.717492
4 leisure 26-30 0.649013
In [ ]:
grouped_category_age = df_age_category_df.groupby(['range', 'ground_truth_category']).mean()
pivot_table_category_age = grouped_category_age.pivot_table(index='ground_truth_category', columns='range', values='positive')
sns.heatmap(pivot_table_category_age, annot=True, cmap="YlGnBu")
Out[ ]:
<Axes: xlabel='range', ylabel='ground_truth_category'>

📒It is not difficult for us to find in the heat map after weighting the emotional scoring model, whether it is 16-20, 21-25, 26-30, 36-40, 41-45, 46-50, 51-55, 56-60 age range Bonding has the strongest correlation, but based on the statistics of the number of happy moments we obtained above, "the biggest source of happiness for married people is affections" and "the biggest source of happiness for single people is achievement". These results are contrary to each other, so we judge this The emotional scoring model may not be suitable for our data. We can try more emotional scoring models in the future, such as roberta, spaCy, etc.

📒 From here we can be more confident that the emotion scoring model we chose has increased the outliers of the data to a certain extent.

When using sentiment models to score data and then creating box plots for weighted statistics, the presence of a large number of outliers can occur for several reasons:

  1. Skewed Data: The underlying data might be skewed. Sentiment scores often have a non-normal distribution, especially in datasets with diverse expressions of sentiment that can lead to a heavy concentration of scores at one end of the spectrum.

  2. Sensitive Scoring Model: The sentiment model used for scoring might be very sensitive to certain keywords or phrases, causing a wider dispersion of sentiment scores that are not representative of the general sentiment.

  3. Diverse Sentiment Expression: People express sentiments in very different ways. Some expressions may be very mild, while others are extremely positive or negative, which can lead to scores that are significantly higher or lower than the median.

  4. Weighting Issues: If the statistical weighting is not applied correctly or the weights themselves are not representative of the population, it can distort the sentiment scores and lead to outliers.

  5. Data Entry Errors: Sometimes, outliers are simply the result of data entry errors or anomalies in the data collection process.

  6. Narrow Interquartile Range (IQR): A box plot displays outliers based on the IQR. If the IQR is narrow due to a large clustering of data within a specific sentiment score range, then even moderately deviant scores can appear as outliers.

To address this, you might consider the following approaches:

  • Data Cleaning: Ensure that the data is cleaned properly to remove any errors that might cause outliers.

  • Model Calibration: Adjust the sentiment model to be less sensitive to outliers or to better accommodate the range of sentiment expressions in your data.

  • Analysis of Outliers: Conduct a thorough analysis of the outliers to understand why they are occurring. This could involve a qualitative review of the high and low sentiment texts.

  • Robust Statistical Methods: Use statistical methods that are less affected by outliers, or adjust the definition of outliers in the context of your data.

Affections_Based Clustering¶

In [1]:
import torch
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

from tqdm.notebook import tqdm

import numpy as np
from scipy.spatial.distance import cdist
In [4]:
sementic_df = pd.read_csv("/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/whole/sementic_cleaned_hm.csv")
In [9]:
demographic_csv_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/data/demographic.csv"
demographic_df = pd.read_csv(demographic_csv_path)
demographic_df.head()
Out[9]:
wid age country gender marital parenthood
0 1 37.0 USA m married y
1 2 29.0 IND m married y
2 3 25 IND m single n
3 4 32 USA m married y
4 5 29 USA m married y
In [10]:
df_ground_truth = sementic_df.dropna(subset=['ground_truth_category'])
In [11]:
merge_df = df_ground_truth.merge(demographic_df,on='wid')
len(merge_df)
Out[11]:
14125
In [12]:
merge_df.head()
Out[12]:
Unnamed: 0 hmid wid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category positive negative neutral age country gender marital parenthood
0 3 27676 206 24h We had a serious talk with some friends of our... We had a serious talk with some friends of our... True 2 bonding bonding 0.954003 0.044106 0.001891 28 DNK f married n
1 21745 49534 206 24h I bought my wedding shoes! I bought my wedding shoes! True 1 achievement achievement 0.221079 0.271567 0.507355 28 DNK f married n
2 24645 52451 206 24h My new watch arrived in the mail! My new watch arrived in the mail! True 1 achievement achievement 0.146610 0.213970 0.639420 28 DNK f married n
3 56163 84125 206 3m We watched Rocky Horror Picture Show for the f... We watched Rocky Horror Picture Show for the f... True 1 leisure leisure 0.717492 0.189807 0.092702 28 DNK f married n
4 57156 85125 206 3m We played Mario Kart while drunk. We played Mario Kart while drunk. True 1 leisure leisure 0.649013 0.241162 0.109825 28 DNK f married n
In [13]:
merge_df = merge_df[merge_df['country']=='USA']
merge_df.head()
Out[13]:
Unnamed: 0 hmid wid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category positive negative neutral age country gender marital parenthood
16 32 27705 5732 24h I picked my daughter up from the airport and w... I picked my daughter up from the airport and w... True 1 bonding affection 0.974089 0.016479 0.009433 64 USA f divorced y
17 42 27715 2272 24h when i received flowers from my best friend when i received flowers from my best friend True 1 bonding bonding 0.987854 0.000706 0.011440 27 USA m single n
18 8526 36258 2272 24h while meeting a friend i laughed so much that ... while meeting a friend i laughed so much that ... True 1 bonding bonding 0.970213 0.015644 0.014143 27 USA m single n
19 14339 42099 2272 24h i bought a new car for which i saved for a ver... i bought a new car for which i saved for a ver... True 1 achievement achievement 0.789608 0.117012 0.093379 27 USA m single n
20 50 27723 389 24h I went shopping I went shopping True 1 leisure leisure 0.126672 0.128704 0.744623 61 USA f married y
In [14]:
temp_df = merge_df[['wid','ground_truth_category','positive']]
In [15]:
temp_df.head()
Out[15]:
wid ground_truth_category positive
16 5732 bonding 0.974089
17 2272 bonding 0.987854
18 2272 bonding 0.970213
19 2272 achievement 0.789608
20 389 leisure 0.126672
In [16]:
temp_df['ground_truth_category'].unique()
Out[16]:
array(['bonding', 'achievement', 'leisure', 'affection',
       'enjoy_the_moment', 'exercise', 'nature'], dtype=object)
In [17]:
temp_df_grp = temp_df.groupby(['wid'])
temp_df_grp
Out[17]:
<pandas.core.groupby.generic.DataFrameGroupBy object at 0x15c1c4950>
In [18]:
records = []

for name,group in tqdm(temp_df_grp):
    rec = {
        "wid":name
    }
    arr = []
    for category in ['bonding', 'achievement', 'leisure', 'affection','enjoy_the_moment', 'exercise', 'nature']:
#         print(len(group[group['ground_truth_category']==category]))
        if (len(group[group['ground_truth_category']==category])> 0):
#             print(f"mean of {category} = {group[group['ground_truth_category']==category]['positive'].mean()}")
            arr.append(group[group['ground_truth_category']==category]['positive'].mean())
        else:
            arr.append(0)
            
    rec['data'] = arr
    records.append(rec)

  0%|          | 0/4644 [00:00<?, ?it/s]
/Users/chenjianfeng/anaconda3/lib/python3.11/site-packages/tqdm/std.py:1182: FutureWarning: In a future version of pandas, a length 1 tuple will be returned when iterating over a groupby with a grouper equal to a list of length 1. Don't supply a list with a single grouper to avoid this warning.
  for obj in iterable:
In [19]:
workers_df = pd.DataFrame(records)
workers_df.head()
Out[19]:
wid data
0 1 [0, 0.137100713327527, 0, 0.4215105965733528, ...
1 4 [0, 0.29191372791926057, 0.30610373616218567, ...
2 5 [0, 0.9422732506479535, 0.9660078287124634, 0....
3 7 [0, 0.9755759239196777, 0, 0.6728441839416822,...
4 9 [0, 0.682466983795166, 0.7587945759296417, 0.3...
In [20]:
workers_df['data'].tolist()
Out[20]:
[[0, 0.137100713327527, 0, 0.4215105965733528, 0, 0, 0],
 [0,
  0.29191372791926057,
  0.30610373616218567,
  0.5476994067430496,
  0.9841907620429992,
  0,
  0],
 [0,
  0.9422732506479535,
  0.9660078287124634,
  0.7193172574043274,
  0.863121509552002,
  0,
  0],
 [0, 0.9755759239196777, 0, 0.6728441839416822, 0, 0, 0],
 [0,
  0.682466983795166,
  0.7587945759296417,
  0.3264326453208923,
  0,
  0,
  0.3608618378639221],
 [0.999874711036682,
  0.8684602975845337,
  0,
  0.8958892822265625,
  0,
  0.1307117491960525,
  0],
 [0,
  0.6169674638658762,
  0.7633823752403259,
  0.6480808854103088,
  0.1526822745800018,
  0.6617514193058014,
  0.25640395283699036],
 [0.9975021481513976, 0.9728579223155975, 0, 0.9819894518171038, 0, 0, 0],
 [0, 0.6628201588988304, 0, 0.8284440040588379, 0.3824873268604278, 0, 0],
 [0.7689815759658813,
  0.3922153562307358,
  0.2177372723817825,
  0.3639296293258667,
  0.8849262595176697,
  0,
  0],
 [0.9059566855430603,
  0.7955472022294998,
  0,
  0.840456485748291,
  0.5624952912330627,
  0,
  0.0938845723867416],
 [0.8958899080753326, 0.26298290491104126, 0, 0.18592155724763865, 0, 0, 0],
 [0.9556574622790018,
  0.6089979360500971,
  0.799452930688858,
  0,
  0.6815130323171615,
  0,
  0],
 [0, 0.46578184288481006, 0, 0.6745045185089111, 0, 0, 0],
 [0.9483643770217896,
  0.40721282901035416,
  0.472448468208313,
  0,
  0.6506055692831675,
  0,
  0],
 [0.9301214416821798,
  0.8769465088844299,
  0.9812466502189636,
  0.0685957819223404,
  0,
  0.3826212584972381,
  0],
 [0.4926667511463165,
  0.4713267336289088,
  0,
  0.5170791536569594,
  0.296459436416626,
  0,
  0],
 [0,
  0.5040227721134821,
  0.2419122159481048,
  0.8007865846157074,
  0.6226471761862437,
  0,
  0],
 [0.8374781608581543,
  0.2978727122147878,
  0.5873757004737854,
  0.2808998748660087,
  0.8766465187072754,
  0,
  0],
 [0, 0.4748096466064453, 0.7896565794944763, 0, 0.964567005634308, 0, 0],
 [0.9215424458185831, 0.6832125782966614, 0, 0.5663713216781616, 0, 0, 0],
 [0,
  0.6933804601430893,
  0,
  0.39410303533077234,
  0.9034669548273087,
  0,
  0.8658231496810913],
 [0.9999866485595704,
  0.5502831128736337,
  0,
  0,
  0.5073545873165131,
  0.185338944196701,
  0],
 [0,
  0.7544189356267452,
  0.7007513493299484,
  0.4470658302307129,
  0.8755398094654083,
  0,
  0],
 [0.988240122795105,
  0.415598937869072,
  0,
  0.9659907817840576,
  0.9083369970321655,
  0.948187291622162,
  0],
 [0.9928910732269288, 0.754799447953701, 0, 0.5756111547350884, 0, 0, 0],
 [0.996465802192688,
  0.8663649260997772,
  0,
  0.9230564832687378,
  0.988989531993866,
  0,
  0],
 [0.239871472120285,
  0.459239199757576,
  0,
  0.5011677853763103,
  0,
  0.1856067031621933,
  0],
 [0.8944230079650879, 0.733821451663971, 0, 0, 0.731142520904541, 0, 0],
 [0.9703196287155152,
  0,
  0.625187873840332,
  0.7831381956736246,
  0.9231996238231659,
  0.1033431366086006,
  0],
 [0, 0, 0, 0.6917418837547302, 0.7850021762507302, 0.0383162312209606, 0],
 [0.79929518699646,
  0.7108263969421387,
  0.6523396223783493,
  0.3196532428264618,
  0.7945820093154907,
  0,
  0],
 [0.8192546740174294,
  0.8645654320716858,
  0.5079385042190552,
  0.7089867293834686,
  0,
  0,
  0],
 [0.6028576493263245,
  0.46833112090826035,
  0,
  0.2387157082557678,
  0,
  0.2868005037307739,
  0],
 [0.8783846696217855, 0.2565247118473053, 0, 0.591572953077654, 0, 0, 0],
 [0.9891067147254944,
  0.9434030652046204,
  0.8670592308044434,
  0.7326608151197433,
  0.0796752274036407,
  0,
  0],
 [0.828554759422938,
  0.7491686791181564,
  0.9996336698532104,
  0.3036886490881443,
  0.755229115486145,
  0,
  0.9890613555908204],
 [0, 0.29721872955560685, 0, 0.5482053607702255, 0, 0, 0],
 [0.9569055438041688,
  0.7566386163234711,
  0,
  0.720783531665802,
  0.8449609676996866,
  0,
  0],
 [0.9926135540008544,
  0.3399884030222892,
  0.1189094111323356,
  0.541741356253624,
  0.3728630542755127,
  0,
  0.7974621057510376],
 [0.9968536496162416,
  0.4969925781091054,
  0.6155764535069466,
  0.1455091387033462,
  0.1779616773128509,
  0,
  0],
 [0, 0.6371815800666809, 0.9668627977371216, 0.5996158868074417, 0, 0, 0],
 [0, 0.9189724624156952, 0, 0.3418990690261125, 0, 0, 0],
 [0.9984659850597382, 0.21195042133331293, 0.5038295984268188, 0, 0, 0, 0],
 [0,
  0.6322824135422707,
  0.87706059217453,
  0.682238906621933,
  0.961948424577713,
  0.9209374189376832,
  0],
 [0,
  0.6114794611930847,
  0.98650324344635,
  0.4198198430240154,
  0.4572100043296814,
  0,
  0],
 [0.8902253210544586, 0.34767340496182436, 0, 0.8208476305007935, 0, 0, 0],
 [0, 0.4214778078926934, 0.49575117230415344, 0, 0, 0, 0.7239409685134888],
 [0,
  0.0988653376698494,
  0.7525047461191813,
  0.5598754733800888,
  0.8871830105781555,
  0,
  0.4831818044185638],
 [0.4350074231624603,
  0.7171584644487926,
  0.022810960188508,
  0,
  0.7467517852783203,
  0,
  0.7174817472696304],
 [0, 0, 0.9738746881484984, 0, 0.9920459985733032, 0, 0],
 [0.6648801267147064,
  0.6031370013952255,
  0,
  0.40263592700163525,
  0.2723465263843536,
  0,
  0],
 [0, 0.11487837508320806, 0.7907939553260803, 0.17696680980069296, 0, 0, 0],
 [0,
  0.5146242141723633,
  0,
  0.1753241841991742,
  0.2968374788761139,
  0,
  0.6932640075683594],
 [0.650206983089447,
  0.23416271805763242,
  0.4285113364458084,
  0.7685751914978027,
  0.4929863587021827,
  0,
  0],
 [0.9458845257759094,
  0.9476355910301208,
  0,
  0.8945919275283813,
  0.6802728772163391,
  0,
  0],
 [0, 0.17768862595160798, 0, 0.22141575174672262, 0, 0, 0],
 [0, 0.45381397008895874, 0, 0.3482124879956245, 0.5118028372526169, 0, 0],
 [0,
  0.5522683560848236,
  0.7328639030456543,
  0.300720501691103,
  0.3928524851799011,
  0,
  0],
 [0.4916459918022156,
  0.5599977572758993,
  0.7413403987884521,
  0.435212604701519,
  0.8937118053436279,
  0,
  0],
 [0,
  0.3788133561611175,
  0.5468888580799102,
  0.2893667891621589,
  0.7007676362991333,
  0.921634554862976,
  0],
 [0.7398422956466675,
  0,
  0.5515993013978004,
  0.3945907255013783,
  0.6331590414047241,
  0,
  0],
 [0.9644231001536051, 0.47703437010447186, 0, 0.3521240353584289, 0, 0, 0],
 [0, 0.8495193421840668, 0, 0.3973444650570552, 0, 0, 0],
 [0, 0.46979445219039917, 0, 0.4651801586151123, 0.4979938517014186, 0, 0],
 [0.7197631001472473,
  0.2750847302377224,
  0.5360426306724548,
  0.0950754359364509,
  0,
  0,
  0.3229615315794945],
 [0, 0.7172285914421082, 0.1054267138242721, 0, 0, 0, 0.9700478315353394],
 [0, 0.6648417413234711, 0, 0.26936504803597927, 0, 0, 0],
 [0,
  0.5346575975418091,
  0.4748132824897766,
  0.8278654038906097,
  0.9479748606681824,
  0,
  0.9699013829231262],
 [0,
  0.8047166168689728,
  0,
  0,
  0.974994421005249,
  0.8144822120666504,
  0.9963191747665404],
 [0, 0.6246875077486038, 0, 0.19577843944231665, 0, 0, 0],
 [0.8359584957361221,
  0.591045469045639,
  0.22885722666978836,
  0,
  0,
  0,
  0.1401931345462799],
 [0.8999206125736237,
  0.34735628465811413,
  0,
  0.45939776860177517,
  0.9970507621765136,
  0,
  0],
 [0.9001186490058899, 0.2257867008447647, 0.0536536835134029, 0, 0, 0, 0],
 [0.9446530342102052,
  0,
  0,
  0.7486021518707275,
  0.4959306120872497,
  0,
  0.8014942407608032],
 [0.8392297029495239,
  0.3910011649131775,
  0.37254180759191513,
  0.5677797198295593,
  0,
  0,
  0],
 [0, 0.3474336490035057, 0, 0.7773531153798103, 0.5059447884559631, 0, 0],
 [0, 0, 0.3284018412232399, 0.25027199275791645, 0.2157990038394928, 0, 0],
 [0.6435366868972778, 0, 0, 0.47761650880177814, 0, 0, 0.8055709004402161],
 [0, 0.579346239566803, 0, 0.8424153923988342, 0, 0, 0],
 [0, 0.3060519084334373, 0.8512274026870728, 0, 0, 0, 0],
 [0.6645182967185974, 0.7834499080975851, 0, 0.6732126375039419, 0, 0, 0],
 [0,
  0.3370041251182556,
  0.7473838150501251,
  0.1194042563438415,
  0.2176694422960281,
  0,
  0.1381328850984573],
 [0.8452879985173544,
  0.3196448404341936,
  0.8344496488571167,
  0.2587212920188904,
  0,
  0,
  0],
 [0,
  0.2357016056776046,
  0.8540000518163046,
  0.25071811676025385,
  0.8572898507118225,
  0,
  0],
 [0.8726728558540344, 0.329301701237758, 0, 0, 0, 0, 0],
 [0.7184855937957764,
  0.2704422175884247,
  0,
  0.3286152631044387,
  0.5623554686705271,
  0,
  0],
 [0,
  0.43621434768040973,
  0.4896071553230285,
  0.5636279359459876,
  0.9276829361915588,
  0,
  0],
 [0.4683143546183903,
  0.698274552822113,
  0.8982224464416504,
  0.5514293797314167,
  0,
  0.987236499786377,
  0],
 [0.9644092321395874, 0.8495615273714066, 0, 0.8226788341999054, 0, 0, 0],
 [0.7226612567901611,
  0.1379027366638183,
  0,
  0.94182687997818,
  0,
  0.3588936987022559,
  0],
 [0.9578983783721924, 0.929506242275238, 0, 0.3568437462672591, 0, 0, 0],
 [0.8145684003829956,
  0.3032463043928146,
  0,
  0.7205374588569006,
  0.2029759734869003,
  0.0958877354860305,
  0],
 [0.8606164753437042,
  0.3879231661558151,
  0,
  0.2935197651386261,
  0.5523084998130798,
  0,
  0],
 [0,
  0.8642290234565735,
  0,
  0.3759870618581772,
  0.2835976481437683,
  0,
  0.545119047164917],
 [0,
  0.4937963187694549,
  0,
  0.36191037297248835,
  0,
  0.2885719835758209,
  0.3577573895454407],
 [0.4901515692472458,
  0.7270233631134033,
  0,
  0.18692402401939032,
  0.9420002102851868,
  0,
  0],
 [0.9879974126815796,
  0,
  0.9021123349666595,
  0.747507855296135,
  0.9284858902295431,
  0.8776967525482178,
  0.9994885921478271],
 [0, 0.4052640981972217, 0.9969916343688964, 0.6238415514429411, 0, 0, 0],
 [0.8169680833816528,
  0.8370913167794546,
  0,
  0.768219788869222,
  0,
  0,
  0.7710949778556824],
 [0.9829716086387634, 0.15124856680631632, 0, 0, 0, 0.0265612136572599, 0],
 [0.0693808421492576, 0.5253352671861649, 0, 0.3593113929033279, 0, 0, 0],
 [0.82593834400177,
  0.3369923746213317,
  0,
  0,
  0.991141676902771,
  0,
  0.34272746245066327],
 [0, 0.26349432642261184, 0, 0.1301777213811874, 0.5117710530757904, 0, 0],
 [0, 0.2685096887871623, 0, 0.0737054869532585, 0, 0, 0],
 [0.8726977904637655, 0.3881535828113556, 0, 0.5879924595355988, 0, 0, 0],
 [0.8911462426185608,
  0.4691902697086334,
  0.5983064949512482,
  0.1079471632838249,
  0,
  0,
  0],
 [0.6712708324193954,
  0.6674621313810348,
  0.5995742082595825,
  0.7495078444480896,
  0,
  0,
  0],
 [0, 0.5911137089133263, 0, 0.9157728850841522, 0.8018226027488708, 0, 0],
 [0, 0.37774621446927387, 0, 0.6061648726463318, 0, 0, 0],
 [0, 0.5572262480854988, 0.3542337417602539, 0.4738970827311277, 0, 0, 0],
 [0, 0.544276624917984, 0.0238804724067449, 0, 0, 0, 0],
 [0.9548856168985367, 0, 0, 0.7844707022110621, 0, 0.9943504929542542, 0],
 [0, 0.4298538684844971, 0.2555645555257797, 0, 0, 0, 0],
 [0,
  0.5001437415679296,
  0.2018689513206482,
  0.9052593410015106,
  0.711388885974884,
  0,
  0],
 [0.8481751084327698,
  0.2106828242540359,
  0.1391331851482391,
  0.17104310914874074,
  0.8073089718818665,
  0,
  0],
 [0.864166647195816,
  0,
  0.48734255135059357,
  0,
  0.6239410638809204,
  0.5981636941432953,
  0],
 [0.9122249633073807,
  0.1571846157312393,
  0.58230921626091,
  0.3338982164859772,
  0.7135291695594788,
  0,
  0.9963982105255128],
 [0, 0.54918172955513, 0, 0, 0.7919442057609558, 0, 0],
 [0.9926827748616537,
  0.42959941178560257,
  0.363324910402298,
  0.647139290968577,
  0.9718157052993774,
  0,
  0],
 [0.9142643213272095,
  0.5984932680924734,
  0.081913910806179,
  0.2149767875671386,
  0,
  0.0529461875557899,
  0],
 [0.5008736634626985, 0.12081726330022013, 0, 0, 0.3371348679065704, 0, 0],
 [0.7376567870378494,
  0.2076014926036199,
  0.8949919939041138,
  0.15126771790285906,
  0.6624586582183838,
  0,
  0],
 [0.9660285115242004,
  0.7882741093635559,
  0,
  0.7800138294696808,
  0.9869790077209472,
  0,
  0],
 [0.9972072243690492, 0.9734096527099608, 0, 0.8973030209541321, 0, 0, 0],
 [0, 0.39572038253148395, 0, 0.5058121494948864, 0.4436399936676025, 0, 0],
 [0.7357881516218185, 0, 0, 0, 0.8267797827720642, 0.2264408841729164, 0],
 [0,
  0.4634135738015175,
  0,
  0.8460221886634827,
  0.7346308728059133,
  0,
  0.9397295117378236],
 [0.9948845505714417, 0.700884997844696, 0, 0.5673792362213135, 0, 0, 0],
 [0, 0.10081899911165235, 0, 0, 0, 0, 0],
 [0, 0, 0.1861652582883834, 0.7361492365598679, 0.0097096590325236, 0, 0],
 [0, 0.528712160885334, 0.19506651908159256, 0.41507573425769806, 0, 0, 0],
 [0, 0.5574690699577332, 0.111922986805439, 0, 0, 0, 0],
 [0.8509494185447692,
  0.5611602465311686,
  0.9535676538944244,
  0,
  0.3331254422664642,
  0,
  0],
 [0.7822547256946564,
  0.49072160199284554,
  0,
  0.4295258820056915,
  0.9813901782035828,
  0,
  0.1755695044994354],
 [0.804538905620575, 0.3813246600329876, 0, 0, 0, 0, 0],
 [0, 0.22208910435438156, 0.6472277045249939, 0.6127094253897667, 0, 0, 0],
 [0,
  0.7202769368886948,
  0,
  0.8603561595082283,
  0.3729786276817322,
  0,
  0.4384655952453613],
 [0, 0.28686569631099695, 0, 0.393586240708828, 0, 0, 0],
 [0, 0.9929047226905824, 0, 0.5404271334409714, 0.8398195505142212, 0, 0],
 [0.9185660779476166,
  0.21479429677128792,
  0.6849555373191833,
  0,
  0.1801740229129791,
  0.035308189690113,
  0],
 [0.9888916015625, 0.8079119324684143, 0, 0, 0.4102725386619568, 0, 0],
 [1.0, 0.546635564416647, 0.9649159908294678, 0, 0, 0, 0],
 [0,
  0.39905660537381965,
  0.6326859444379807,
  0.0689348056912422,
  0.2303346544504165,
  0,
  0.1327121071517467],
 [0.9231947958469391,
  0.5340053041776022,
  0.1690488308668136,
  0,
  0.9987395405769348,
  0,
  0.6432285855213801],
 [0.8562845885753632,
  0,
  0.1786884516477584,
  0.5510889068245888,
  0,
  0,
  0.2550312280654907],
 [0, 0.5063136617342631, 0.8507179617881775, 0, 0.5647611916065216, 0, 0],
 [0.9978983402252196, 0.5523485988378525, 0, 0, 0.9821915030479432, 0, 0],
 [0,
  0.3176757246255874,
  0.22325392067432398,
  0,
  0.3558525890111923,
  0,
  0.1815146654844284],
 [0,
  0.3919810702403386,
  0.5349002331495285,
  0.44712087512016296,
  0.7652405500411987,
  0,
  0],
 [0, 0.9978742599487304, 0, 0.9922943115234376, 0.9901090264320374, 0, 0],
 [0, 0.5994426608085632, 0.4309671223163605, 0, 0, 0, 0],
 [0.9978166818618774, 0, 0.1292245537042617, 0.8263155966997147, 0, 0, 0],
 [0, 0.9274757504463196, 0, 0.45805882786711055, 0, 0, 0],
 [0, 0.2799848228693008, 0, 0.6328828483819962, 0.8793724477291107, 0, 0],
 [0,
  0.3456697314977646,
  0.6213160753250122,
  0.14388381130993358,
  0,
  0.3303889334201813,
  0],
 [0, 0.6092732071876525, 0.6408600211143494, 0, 0.5972318798303604, 0, 0],
 [0, 0.5054946273565293, 0.30851583182811737, 0.6773099005222321, 0, 0, 0],
 [0.802270770072937,
  0,
  0,
  0.4464475214481354,
  0,
  0.23464918881654734,
  0.7581990957260132],
 [0, 0.48001097366213796, 0, 0, 0, 0, 0],
 [0.8419656157493591,
  0.5563418753445148,
  0.5241226106882094,
  0.0792440921068191,
  0,
  0.0346792861819267,
  0],
 [0,
  0.2932811876138051,
  0.0592299699783325,
  0.2059565037488937,
  0.2641794979572296,
  0,
  0],
 [0.7979507247606913, 0.4465004950761795, 0, 0.2741715628653764, 0, 0, 0],
 [0, 0.5791000425815582, 0.1556778848171234, 0, 0, 0.0394679866731166, 0],
 [0.7704485356807709,
  0,
  0.9513320326805115,
  0.737192710240682,
  0.4819197356700897,
  0,
  0],
 [0.9781719446182252, 0.5962215065956116, 0, 0.32303112745285034, 0, 0, 0],
 [0.8827366828918457,
  0.3077099621295929,
  0.12230298668146128,
  0.21348782380421952,
  0.4664995148777962,
  0.2056480646133422,
  0],
 [0.9893763065338136, 0.8834720452626547, 0, 0.45202524463335675, 0, 0, 0],
 [0,
  0.3214727317293485,
  0.4550713499387105,
  0.3499137759208679,
  0.13750996937354404,
  0.087956003844738,
  0],
 [0, 0.4329731166362762, 0.0731195881962776, 0.4842660203576088, 0, 0, 0],
 [0.9456060826778412,
  0.6813242435455322,
  0.8390568792819977,
  0.122007317841053,
  0,
  0,
  0],
 [0, 0.265958309173584, 0.9739426374435424, 0.6485097656647364, 0, 0, 0],
 [0,
  0.73462975025177,
  0.777972400188446,
  0.5994932120665908,
  0.835498591264089,
  0,
  0],
 [0.9284268021583557,
  0.6166851563113076,
  0,
  0.4593763351440429,
  0,
  0,
  0.9490879774093628],
 [0, 0.7111337880293528, 0, 0.43853725492954254, 0, 0, 0],
 [0,
  0.1968529919783274,
  0.2112407684326172,
  0.6509882211685181,
  0.1863065958023071,
  0,
  0],
 [0, 0.1307744979858398, 0, 0.29106123248736065, 0.9963919520378112, 0, 0],
 [0.9863648116588593,
  0.23971940949559206,
  0,
  0.14967084676027295,
  0.7432439724604288,
  0,
  0],
 [0.8062226921319962,
  0.3319841573635737,
  0.8854308128356934,
  0,
  0.7867286801338196,
  0,
  0],
 [0, 0.6324196060498556, 0, 0.1613005548715591, 0, 0, 0],
 [0.8679644465446472,
  0.33648414909839625,
  0,
  0.3731407821178436,
  0.1517722904682159,
  0,
  0],
 [0.979785978794098,
  0.19720972329378123,
  0,
  0.1593278646469116,
  0.2375853657722473,
  0,
  0.0748015940189361],
 [0, 0.5664648314317068, 0, 0, 0.7428339123725891, 0, 0],
 [0,
  0.6432402729988098,
  0.1605892181396484,
  0.0702291652560234,
  0,
  0.293784886598587,
  0],
 [0.8213959336280823,
  0.332244873046875,
  0.3458787649869919,
  0.3156004548072815,
  0,
  0,
  0],
 [0.8309128681818644, 0.1393556954960028, 0, 0.4541729435324669, 0, 0, 0],
 [0.8560372591018677, 0, 0.9172278046607972, 0.8311874747276307, 0, 0, 0],
 [0.9755725860595704,
  0.4749549627304077,
  0,
  0.38686052922691616,
  0.5019310712814331,
  0,
  0],
 [0, 0.9605554789304733, 0, 0, 0.9690437763929367, 0, 0],
 [0,
  0.41553446277976036,
  0.1443416327238083,
  0,
  0.5721946954727173,
  0.5461642146110535,
  0],
 [0, 0.7363198697566986, 0, 0.16078752279281616, 0, 0, 0],
 [0.99739408493042,
  0.6936240345239639,
  0.1596115082502365,
  0.9764448602994283,
  0.872135728597641,
  0,
  0],
 [0, 0, 0.9670518636703492, 0.28878501057624817, 0, 0, 0],
 [0,
  0.17779475077986712,
  0.6213883757591248,
  0.3416204284876585,
  0.8504824638366699,
  0,
  0],
 [0.8007368743419647,
  0.2689311727881431,
  0,
  0.48373682690518244,
  0.0472000166773796,
  0,
  0],
 [0.9556874632835388, 0.6609023064374924, 0, 0.19704150408506388, 0, 0, 0],
 [0, 0.7767488360404968, 0, 0.32496565394103527, 0, 0, 0.8497326374053955],
 [0.8804509441057841, 0.0156829394400119, 0, 0, 0.8238566517829895, 0, 0],
 [0.6733457297086716,
  0.4426014522711436,
  0.7504709959030151,
  0.4655395448207855,
  0.5211932361125946,
  0,
  0],
 [0, 0.5713000893592834, 0.925439178943634, 0, 0.5862599015235901, 0, 0],
 [0,
  0.33422468105951947,
  0.5392124056816101,
  0.5470330230891705,
  0.7335191170374552,
  0,
  0],
 [0.899233877658844,
  0.7645756999651591,
  0.6326782703399658,
  0.35950786620378494,
  0.34087981283664703,
  0.397239625453949,
  0.4850168824195862],
 [0, 0.25545313830176986, 0, 0, 0, 0, 0],
 [0.9998890161514282, 0.29533302038908005, 0, 0.5821440815925598, 0, 0, 0],
 [0.3749108016490936,
  0.24348757664362586,
  0.20523162372410295,
  0,
  0,
  0,
  0.6003963351249695],
 [0.9995214939117432,
  0.9995242357254028,
  0,
  0.8388077616691589,
  0.9899723529815674,
  0,
  0],
 [0.9972674250602722,
  0.7570285797119141,
  0.3269025608897209,
  0,
  0.9994139671325684,
  0,
  0],
 [0, 0, 0, 0.22428161718628623, 0, 0.5461642146110535, 0],
 [0, 0.44836642444133756, 0.22209590859711165, 0, 0, 0, 0],
 [0, 0.5533560588955879, 0, 0.5092022672295571, 0.5816636085510254, 0, 0],
 [0.7984632253646851,
  0.46863795071840286,
  0,
  0.9956745505332948,
  0.5651941299438477,
  0,
  0],
 [0.9982308745384216,
  0.7287799417972565,
  0.6130238994956017,
  0.5971056322256724,
  0.1060821935534477,
  0,
  0],
 [0,
  0.41119364351034154,
  0.3150951266288757,
  0.1815596781671047,
  0,
  0.6822801232337952,
  0],
 [0, 0.3347357213497162, 0, 0.4595852792263031, 0.2344984114170074, 0, 0],
 [0.9998984336853028,
  0,
  0.7084952592849731,
  0.079223483800888,
  0.9805095195770264,
  0,
  0],
 [0, 0, 0, 0.8125068135559559, 0, 0, 0],
 [0.9835769534111024,
  0.467913955450058,
  0.40767569839954376,
  0,
  0.623910998304685,
  0,
  0],
 [0.7439854145050049, 0.8510943651199341, 0, 0.3962196658054988, 0, 0, 0],
 [0.49916059523820877,
  0.8659530878067017,
  0.6360049247741699,
  0.5824673771858215,
  0.5845181345939636,
  0,
  0.7579456567764282],
 [0.1811566352844238,
  0.7207683722178141,
  0.6099719405174255,
  0.7091333121061325,
  0.2971775233745575,
  0.1764267385005951,
  0],
 [0.7922612031300863,
  0.19624385610222808,
  0.6904131174087524,
  0.15270481258630744,
  0.8367864191532135,
  0.6318855285644531,
  0],
 [0.0865812599658966,
  0.3893836960196495,
  0.7687567174434662,
  0.7240468561649323,
  0.8669095039367676,
  0,
  0.2205751687288284],
 [0.9969874620437622, 0.6258768935998281, 0, 0, 0.9823463559150696, 0, 0],
 [0.8118579387664795,
  0.8830622434616089,
  0.2105603963136673,
  0.643403947353363,
  0.9464197754859924,
  0,
  0],
 [0.6297754049301147, 0, 0, 0, 0, 0.1636343151330948, 0],
 [0.8564914464950562,
  0.666606068611145,
  0.2102146918574969,
  0.49373987317085266,
  0.1862142235040664,
  0,
  0],
 [0.7557061314582825, 0.4330809433013201, 0, 0.1005442887544632, 0, 0, 0],
 [0, 0.26537132759888965, 0, 0.400124525030454, 0.2108171731233596, 0, 0],
 [0, 0.3123012036085129, 0.3201530873775482, 0.4311063513159752, 0, 0, 0],
 [0, 0, 0, 0, 0.7805843949317932, 0.6172933330138525, 0],
 [0, 0, 0, 0.9873892068862916, 0.6170027852058411, 0, 0],
 [0.6844281256198883,
  0.1958197951316833,
  0,
  0.9949105381965636,
  0,
  0.936664581298828,
  0],
 [0.7379043400287628,
  0.1619765609502792,
  0.9908735752105712,
  0,
  0.48744473854700726,
  0,
  0],
 [0.993450164794922, 0, 0, 0.8544389520372663, 0.9306215047836304, 0, 0],
 [0.6508020162582397,
  0.2492445409297943,
  0.5772055188814799,
  0.3132221270352602,
  0,
  0,
  0],
 [0, 0.107560709118843, 0, 0.5225786268711089, 0, 0, 0],
 [0.8143219351768494, 0, 0, 0.5865894675254821, 0, 0, 0],
 [0.6367020010948181,
  0.834637721379598,
  0.2138813585042953,
  0,
  0.20519730634987352,
  0,
  0],
 [0,
  0.0208768136799335,
  0.7924904227256775,
  0.18323681503534317,
  0.24944016337394714,
  0,
  0],
 [0.9942387342453004,
  0.8171642422676086,
  0.9450928568840027,
  0.8407741189002991,
  0,
  0,
  0],
 [0, 0.5815988692144552, 0, 0, 0.7107625603675842, 0, 0],
 [0, 0.4518919562300046, 0, 0, 0.3204193413257599, 0, 0],
 [0, 0.2056365460157394, 0, 0.3455499112606048, 0, 0, 0],
 [0,
  0.8128872875434657,
  0,
  0.997957706451416,
  0.8848416805267334,
  0.0570146031677722,
  0],
 [0,
  0.420476958155632,
  0.5994284600019455,
  0.8864354491233826,
  0.8968074917793274,
  0,
  0],
 [0.816849946975708, 0.38152294357617694, 0, 0, 0.2478728592395782, 0, 0],
 [0.9950836598873138, 0.9795116583506266, 0, 0, 0.8035786151885986, 0, 0],
 [0, 0.241003930568695, 0, 0.32489597983658314, 0, 0, 0.2975417375564575],
 [0,
  0.3310004025697708,
  0.9985761642456056,
  0.2645876109600067,
  0.7830245196819305,
  0,
  0.9954328536987304],
 [0.7088442742824554,
  0.5205787122249603,
  0.8420835137367249,
  0,
  0.9601686000823976,
  0.6900169849395752,
  0],
 [0, 0, 0.5729622840881348, 0, 0.9634013175964355, 0, 0],
 [0.8381215631961823,
  0,
  0.8683924973011017,
  0,
  0.9665632843971252,
  0.9786466956138612,
  0],
 [0, 0.1856657937169075, 0.1266721487045288, 0.2495082974433899, 0, 0, 0],
 [0, 0.15945752710103986, 0, 0.5166910650829474, 0, 0, 0],
 [0.7219779094060262,
  0.1588892638683319,
  0,
  0.17253943532705301,
  0.9406763911247252,
  0,
  0.737669825553894],
 [0.9934872090816498,
  0.7149418294429779,
  0.8734255433082581,
  0.183787539601326,
  0.7730648517608643,
  0.0482539385557174,
  0],
 [0.988678514957428,
  0.3026688794294993,
  0,
  0.2336050570011139,
  0.35700760781764984,
  0,
  0],
 [0, 0, 0.12303000129759309, 0.3007020056247711, 0, 0, 0],
 [0.9451434016227722,
  0.7562824487686157,
  0.8175825476646423,
  0.5298394560813904,
  0,
  0,
  0.6372030526399612],
 [0, 0.7905397117137909, 0, 0.7027046233415604, 0.3310720026493072, 0, 0],
 [0,
  0.5621652901172638,
  0,
  0.8337553143501282,
  0.7538435856501261,
  0,
  0.5987796485424042],
 [0.8386683464050293, 0.246468611061573, 0, 0.6362367570400238, 0, 0, 0],
 [0.9572337865829468, 0, 0, 0, 0, 0, 0],
 [0, 0.7894273996353149, 0.7770558893680573, 0.7630006549879909, 0, 0, 0],
 [0.5821002125740051,
  0.35385642945766443,
  0.44948211312294006,
  0,
  0.5478093586862086,
  0.87690669298172,
  0],
 [0.7566160261631012,
  0,
  0.6336005330085754,
  0.5626851618289948,
  0.5265098810195923,
  0.3148059844970703,
  0],
 [0,
  0.1069252975285053,
  0.2171537925799687,
  0,
  0.7322218418121338,
  0.099958561360836,
  0],
 [0, 0.1281591951847076, 0.8135654628276825, 0.5783732458949089, 0, 0, 0],
 [0, 0, 0.7666487097740173, 0.5250461008399725, 0.6989677598079046, 0, 0],
 [0, 0.41810733638703823, 0, 0.4004928469657898, 0, 0, 0],
 [0, 0.5602727284034094, 0, 0, 0.0885403081774711, 0, 0],
 [0, 0.2330359667539596, 0, 0.0880937278270721, 0, 0, 0],
 [0, 0.4942365121096372, 0, 0.3730760812759399, 0.1829210072755813, 0, 0],
 [0.9928684234619141, 0.8692396283149719, 0, 0, 0.9359611570835114, 0, 0],
 [0, 0.9447527527809144, 0.709875762462616, 0, 0.8256950676441193, 0, 0],
 [0, 0.9781196117401124, 0, 0.6848847766717275, 0, 0, 0],
 [0, 0.4476963877677917, 0, 0.503273886628449, 0, 0, 0],
 [0.9608362118403116,
  0.5362225882709026,
  0.7349990755319595,
  0,
  0.6177731305360794,
  0,
  0],
 [0.9984353184700012,
  0,
  0.6312538981437683,
  0.5850812122225761,
  0.918816938996315,
  0,
  0],
 [0.9612076282501221, 0.3263781815767288, 0, 0, 0.5791119933128357, 0, 0],
 [0.9940436482429504,
  0.730781614780426,
  0.9449558854103088,
  0,
  0,
  0.2127017825841903,
  0],
 [0, 0.6274682333072027, 0, 0.6888274550437927, 0.7873527407646179, 0, 0],
 [0.8503265182177225,
  0.5099145472049713,
  0,
  0,
  0.942871630191803,
  0.5583181381225586,
  0],
 [0.7112729748090109,
  0.1696243584156036,
  0.6324089169502258,
  0,
  0.9075960516929626,
  0.0961633995175361,
  0],
 [0,
  0.6598126292228699,
  0.6590160727500916,
  0.5899926821390787,
  0.0334092229604721,
  0,
  0.4139151275157928],
 [0.6768554449081421,
  0.6652222126722335,
  0,
  0.5039225518703461,
  0.838440865278244,
  0,
  0.8736395239830017],
 [0.9745277464389801, 0.7421181797981262, 0, 0, 0, 0, 0],
 [0.9343262910842896,
  0.2946454733610153,
  0,
  0.5565507039427757,
  0.4021599590778351,
  0,
  0],
 [0, 0.22429838031530375, 0.5029482841491699, 0.742833286523819, 0, 0, 0],
 [0.995016872882843,
  0.21270858496427533,
  0.498596578836441,
  0.5551681195696195,
  0,
  0,
  0],
 [0, 0.6314344257116318, 0.7965166568756104, 0.5211942031979561, 0, 0, 0],
 [0.7731334865093231, 0, 0.3416183590888977, 0, 0, 0.6009566783905029, 0],
 [0.959713488817215,
  0.6702028065919876,
  0.8838085532188416,
  0.7258712351322174,
  0,
  0,
  0],
 [0, 0.08262122608721251, 0, 0, 0.9779844880104064, 0, 0.5467406660318375],
 [0, 0.978966474533081, 0, 0, 0, 0, 0],
 [0.1223040595650672,
  0.6929327895243963,
  0.4569462835788727,
  0.7847207188606262,
  0,
  0,
  0],
 [0.882153848807017, 0.5621363818645477, 0, 0, 0, 0, 0],
 [0, 0.3586941063404083, 0, 0.381657987833023, 0.5497039929032326, 0, 0],
 [0, 0.497615497559309, 0, 0, 0.978785514831543, 0, 0],
 [0.7790702879428864,
  0.2149482096234957,
  0.3453178480267524,
  0.2670140564441681,
  0.5723929032683372,
  0,
  0],
 [0.6039836406707764,
  0.7528332054615021,
  0.3052937388420105,
  0.6449896395206451,
  0,
  0,
  0.9179275333881378],
 [0,
  0.33526357511679333,
  0.6451239585876465,
  0.5189908295869827,
  0.4281316101551056,
  0.22580205649137497,
  0],
 [0.6477344483137131,
  0.47609639167785645,
  0,
  0.5120636895298958,
  0,
  0.7646875381469727,
  0],
 [0, 0, 0, 0.6064365059137344, 0, 0, 0],
 [0, 0, 0.5158574655652046, 0.4046141505241394, 0.6972938974698385, 0, 0],
 [0.5423881337046623, 0, 0.038616694509983, 0, 0.7057843208312988, 0, 0],
 [0, 0.9402317851781845, 0, 0, 0, 0, 0],
 [0.8040511906147003,
  0.29817384853959084,
  0.7806905110677084,
  0,
  0.7617929577827454,
  0,
  0.1583788990974426],
 [0.9550345540046692,
  0.6622911840677261,
  0.5434426069259644,
  0,
  0.926654577255249,
  0,
  0],
 [0,
  0.7678103446960449,
  0.8268782496452332,
  0.340161383152008,
  0.641480565071106,
  0,
  0],
 [0, 0.9966151118278505, 0, 0.7715120911598206, 0.975661277770996, 0, 0],
 [0.7435134053230286,
  0.4030219515164693,
  0.076835311949253,
  0.5595076282819113,
  0,
  0,
  0],
 [0,
  0.8110315799713135,
  0.3243432641029358,
  0.3337995707988739,
  0.9988631010055542,
  0,
  0.4441543817520141],
 [0.9996440410614014, 0.852225735783577, 0, 0.9803370833396912, 0, 0, 0],
 [0.2524967789649963,
  0.18975691994031266,
  0.5360426306724548,
  0,
  0.9645208716392516,
  0,
  0],
 [0, 0, 0, 0.5206875093281269, 0.1082221716642379, 0, 0],
 [0, 0.523106706639131, 0, 0.4798218235373497, 0.9025421738624572, 0, 0],
 [0.7834787964820862,
  0.8023123294115067,
  0,
  0.8881077766418457,
  0.6748210787773132,
  0,
  0],
 [0, 0.7796804904937744, 0, 0, 0, 0, 0],
 [0,
  0.1838538348674774,
  0.4865972697734833,
  0.5810543100039164,
  0.8425673047701517,
  0,
  0],
 [0.953156352043152, 0.5079969167709351, 0, 0.31162513978779316, 0, 0, 0],
 [0, 0.4157935120165348, 0, 0.0754139795899391, 0, 0.2654641270637512, 0],
 [0, 0, 0, 0, 0.8711217194795609, 0, 0],
 [0.8219998478889465, 0.37386333942413325, 0, 0.2543750703334808, 0, 0, 0],
 [0, 0.1966334730386734, 0, 0, 0, 0, 0],
 [0.8314670026302338, 0.3530103266239166, 0, 0, 0.2253075391054153, 0, 0],
 [0, 0, 0.8220926225185394, 0.34634541906416416, 0.3037208914756775, 0, 0],
 [0.9372914036115011, 0, 0, 0.8409718871116638, 0.5964809060096741, 0, 0],
 [0,
  0.35249607264995575,
  0,
  0.5359532907605171,
  0.6954401135444641,
  0.950758934020996,
  0],
 [0, 0.8744927644729614, 0, 0.910670042037964, 0, 0, 0.4472685158252716],
 [0,
  0.910546600818634,
  0.1028284523636102,
  0,
  0.883188009262085,
  0.0759967043995857,
  0],
 [0,
  0.27520420650641125,
  0.2702395021915436,
  0.3174309954047203,
  0.6858702749013901,
  0,
  0.4024299085140228],
 [0.9251145720481873,
  0,
  0.45697402209043503,
  0.3043459951877594,
  0,
  0,
  0.9620466232299804],
 [0, 0.30384591221809387, 0, 0.3965531438589096, 0, 0, 0],
 [0, 0.3800164759159088, 0, 0, 0.4251860678195953, 0, 0],
 [0.9764615297317504,
  0.4839383413394292,
  0,
  0.7046013474464417,
  0.3556149899959564,
  0,
  0],
 [0, 0, 0.2036140114068985, 0.4817601641019185, 0.152746707201004, 0, 0],
 [0.9710545539855956,
  0.8153036534786224,
  0.6385959386825562,
  0.3077259063720703,
  0.9854795336723328,
  0,
  0],
 [0.6438920845588049,
  0.24616820116837815,
  0,
  0.2717242240905761,
  0.5915159359574318,
  0,
  0],
 [0, 0, 0.6256106495857239, 0.4951717058817546, 0, 0, 0.1851842403411865],
 [0.7007572650909424, 0, 0, 0.37960462272167206, 0, 0, 0],
 [0.9278469681739808, 0.2468920946121215, 0, 0, 0.8570576310157776, 0, 0],
 [0,
  0.5871265828609467,
  0.7223671674728394,
  0.7077268163363138,
  0.6723152101039886,
  0,
  0],
 [0,
  0.5642711073160172,
  0.3435225784778595,
  0.1312782317399978,
  0,
  0.3536067903041839,
  0],
 [0.5162562727928162, 0, 0.8392302393913269, 0.7154431939125061, 0, 0, 0],
 [0.147100567817688,
  0.2298146169632673,
  0.7731201648712158,
  0.1227513402700424,
  0.44094385951757425,
  0,
  0],
 [0, 0.8266580899556478, 0.6149392127990723, 0, 0.8964510560035706, 0, 0],
 [0, 0.7314876914024353, 0, 0.4436020515859127, 0, 0, 0],
 [0, 0.5136096365749836, 0, 0, 0, 0.6927461624145508, 0.140734925866127],
 [0.6238311479489008,
  0.5360317975282669,
  0.8143848180770874,
  0,
  0.8052598237991333,
  0,
  0],
 [0.6015934944152832, 0.5281173586845397, 0, 0, 0, 0, 0],
 [0,
  0.6524821780622005,
  0,
  0.30423002131283283,
  0.2955431640148163,
  0.3832864761352539,
  0],
 [0, 0.955089032649994, 0, 0, 0.3928476870059967, 0, 0.2172146737575531],
 [0, 0.3461666417618592, 0, 0.41486856341362, 0.6018030246098837, 0, 0],
 [0.9369957447052002, 0, 0, 0.1213993784040212, 0, 0, 0],
 [0, 0, 0, 0.8148401578267416, 0, 0, 0],
 [0.9768990874290466, 0.5057251542806626, 0, 0.5789487808942795, 0, 0, 0],
 [0, 0.2821734845638275, 0.3119708299636841, 0, 0.33400761894881725, 0, 0],
 [0, 0.3140790928155184, 0, 0.9671427011489868, 0.6491116285324097, 0, 0],
 [0.8002915382385254,
  0.461325707534949,
  0,
  0.1172638138135274,
  0.6770708858966827,
  0,
  0],
 [0, 0.92870432138443, 0, 0.42716166625420254, 0.906257688999176, 0, 0],
 [0, 0.47557295858860016, 0, 0.3881077151745558, 0, 0, 0],
 [0, 0.9891958236694336, 0, 0.9608165621757508, 0, 0, 0],
 [0.47205908099810284,
  0.7996997237205505,
  0,
  0.31972119584679604,
  0.7787186503410339,
  0,
  0],
 [0, 0, 0, 0.8683852255344391, 0.7349409461021423, 0, 0],
 [0.9999998807907104,
  0.9777829051017762,
  0.9862831234931946,
  0.9999991655349731,
  0,
  0,
  0],
 [0, 0.4845574274659157, 0, 0, 0.0913334637880325, 0, 0],
 [0.9999896287918092, 0, 0, 0.9777585665384928, 0.9608622193336488, 0, 0],
 [0,
  0.5437105745077133,
  0.42627024464309216,
  0.07967353239655489,
  0.5556866526603699,
  0,
  0.4145806431770324],
 [0, 0.4491139352321624, 0.9649006724357604, 0.23558125148216882, 0, 0, 0],
 [0.9962751865386964, 0.41363291442394257, 0, 0.4216185212135315, 0, 0, 0],
 [0, 0, 0, 0.6485932171344757, 0, 0, 0],
 [0.9998890161514282, 0.32824513316154474, 0, 0.2374500781297683, 0, 0, 0],
 [0, 0.452039768298467, 0.8974469304084778, 0, 0, 0, 0],
 [0,
  0.1993871927261352,
  0.0878019407391548,
  0.665984034538269,
  0.1209334135055542,
  0,
  0],
 [0.9384191036224364,
  0.1724011376500129,
  0.2087767869234085,
  0.2537695467472076,
  0,
  0,
  0.7323415875434875],
 [0.960667073726654,
  0.5843788683414459,
  0,
  0,
  0.6256054937839508,
  0.1864766180515289,
  0],
 [0, 0.5168917179107666, 0.07351551577448845, 0, 0.6089346607526144, 0, 0],
 [0, 0, 0, 0.6351051926612854, 0, 0, 0],
 [0, 0.9111154675483704, 0.930750012397766, 0, 0, 0, 0.9875375628471376],
 [0, 0.20818388462066645, 0, 0, 0, 0, 0],
 [0.5600503757596016, 0.0656753927469253, 0, 0, 0.3512012958526611, 0, 0],
 [0.9953336119651794, 0.3678798079490661, 0.8075927495956421, 0, 0, 0, 0],
 [0, 0.3053198680281639, 0, 0.24937367439270014, 0, 0.6856358647346497, 0],
 [0, 0.3840418800711632, 0, 0.2350977826863527, 0, 0, 0],
 [0, 0.6071517467498779, 0, 0, 0.9803429245948792, 0, 0.5567795038223267],
 [0, 0.7852000892162323, 0, 0, 0.4911531507968902, 0, 0],
 [0.95208939909935, 0, 0, 0.6147050062815348, 0, 0.1899990141391754, 0],
 [0.9926849603652954,
  0.343556580444177,
  0,
  0.555364727973938,
  0.2667821049690246,
  0,
  0],
 [0, 0.16216929256916046, 0, 0, 0, 0, 0],
 [0.9573382139205932,
  0.7092676758766174,
  0,
  0.5760601162910461,
  0.1828581740458806,
  0,
  0],
 [0, 0.9991108775138856, 0, 0.35282890250285465, 0.8010286688804626, 0, 0],
 [0, 0, 0, 0.2007298022508621, 0, 0, 0],
 [0, 0.9451653560002645, 0, 0.1803281009197235, 0, 0, 0],
 [0.9988708794116974,
  0.6249582171440125,
  0,
  0.2539060711860657,
  0.9994001388549804,
  0,
  0.3878496587276459],
 [0, 0.7544551491737366, 0, 0.6816867798566818, 0.9977295994758606, 0, 0],
 [0.9029143452644348, 0.3702218234539032, 0, 0.7640600800514221, 0, 0, 0],
 [0.884465217590332,
  0.3094690144062042,
  0.6213160753250122,
  0.2061213850975036,
  0,
  0,
  0],
 [0.9810007810592652, 0, 0, 0.0512773990631103, 0.765964150428772, 0, 0],
 [0, 0.6776813268661499, 0, 0.4881218439550139, 0, 0, 0],
 [0.8814021348953247, 0.43518852690855664, 0, 0, 0.0283249616622924, 0, 0],
 [0, 0, 0.5920616909861564, 0.9890457391738892, 0.4395351409912109, 0, 0],
 [0.8933130204677582, 0, 0, 0.3037295639514923, 0, 0, 0],
 [0, 0, 0, 0, 0, 0, 0.7404778003692627],
 [0, 0, 0, 0.4031233787536621, 0.99640154838562, 0, 0],
 [0, 0.461148053407669, 0, 0.44283581897616386, 0.2058454006910324, 0, 0],
 [0.9220028221607208, 0, 0.6359447240829468, 0.9592930674552916, 0, 0, 0],
 [0, 0, 0, 0.4425569511950016, 0, 0, 0],
 [0.9873189330101012, 0.0060455086641013, 0.8902775347232819, 0, 0, 0, 0],
 [0.9959232807159424, 0.0386448092758655, 0, 0.7156287531057993, 0, 0, 0],
 [0, 0, 0.645861804485321, 0.1106426520273089, 0.9364227652549744, 0, 0],
 [0.129042848944664,
  0.47644665837287903,
  0,
  0.4139335602521896,
  0.1252190023660659,
  0,
  0],
 [0.5930022597312927, 0.698541522026062, 0, 0.1505200117826461, 0, 0, 0],
 [0.9998890161514282, 0.7999073266983032, 0, 0.45566015541553495, 0, 0, 0],
 [0.993931770324707,
  0,
  0.1695068925619125,
  0.6346914768218994,
  0.768495500087738,
  0,
  0],
 [0, 0.24813329676787058, 0.8885292410850525, 0.8364865183830261, 0, 0, 0],
 [0, 0.14843906462192533, 0, 0.1401215940713882, 0, 0, 0.2685529291629791],
 [0.6887931227684021, 0.8852582573890686, 0, 0, 0, 0, 0],
 [0.8686851859092712, 0.43364093080162996, 0, 0.8242748081684113, 0, 0, 0],
 [0,
  0.6010861285030842,
  0,
  0.2368294820189476,
  0.3374196887016296,
  0,
  0.2716951370239258],
 [0.5866780877113342, 0, 0.0362075716257095, 0.10537018378575642, 0, 0, 0],
 [0, 0.6964412331581115, 0, 0.3112541139125824, 0, 0, 0],
 [0, 0.11200514435768123, 0, 0.7785935997962952, 0.8017527858416239, 0, 0],
 [0.7240637540817261, 0, 0, 0.3500118032097816, 0, 0, 0.6677486896514893],
 [0.9916799664497375,
  0.7147151529788971,
  0.7235030879576999,
  0,
  0.9466767311096191,
  0,
  0],
 [0, 0.1857205629348755, 0.6096715927124023, 0, 0, 0, 0],
 [0, 0.606584203739961, 0, 0.4164561927318573, 0.6092907190322876, 0, 0],
 [0.9979648590087892, 0, 0.1264958977699279, 0.3726506332556407, 0, 0, 0],
 [0, 0.5704122185707091, 0, 0.5717292129993439, 0, 0, 0],
 [0, 0, 0.9629878997802734, 0.9613381624221802, 0.8398213187853495, 0, 0],
 [0, 0.6706158071756363, 0, 0, 0.9973130822181702, 0, 0],
 [0.988421638806661, 0, 0.9802175760269164, 0.8149416049321493, 0, 0, 0],
 [0,
  0.37328919768333435,
  0.3806568682193756,
  0.1496919989585876,
  0.3341141045093536,
  0,
  0],
 [0, 0.9679225087165833, 0.8483239710330963, 0.6074412862459818, 0, 0, 0],
 [0.8902291456858317,
  0,
  0,
  0.26659453660249705,
  0,
  0.7322906255722046,
  0.9686283469200134],
 [0.9999173879623412, 0.6506758630275726, 0, 0, 0, 0, 0.9089216589927672],
 [0.72590771317482, 0.4005575142800808, 0, 0.0942002162337303, 0, 0, 0],
 [0.9852933287620544,
  0.6502009630203247,
  0.5659034252166748,
  0.9324586987495422,
  0,
  0,
  0],
 [0, 0, 0, 0.3443641662597656, 0, 0, 0],
 [0.9946546256542206, 0, 0, 0, 0, 0, 0],
 [0.9746590852737428, 0.3960438966751098, 0, 0.6034130652745565, 0, 0, 0],
 [0, 0.4194024329384168, 0.5128115545958281, 0, 0.999326229095459, 0, 0],
 [0.8137806951999664,
  0.3357343425353368,
  0,
  0,
  0.8566827178001404,
  0.9168493747711182,
  0],
 [0.9663036465644836, 0.8545006364583969, 0, 0, 0.8577377398808798, 0, 0],
 [0.9630252718925476, 0, 0, 0.18535096943378446, 0, 0, 0],
 [0.9080576598644257, 0, 0, 0.9467307925224304, 0, 0, 0.9354699850082396],
 [0.997933268547058,
  0.2280149273574352,
  0.7801306843757629,
  0.6138735711574554,
  0,
  0.7431475520133972,
  0],
 [0, 0.09757677714029944, 0.406447172164917, 0.5127186179161072, 0, 0, 0],
 [0.9999897480010986, 0, 0, 0.07559471460990601, 0, 0, 0],
 [0.966854214668274,
  0.3062873780727386,
  0.2078906148672104,
  0,
  0,
  0.919008731842041,
  0.7302050590515137],
 [0, 0, 0.0893199518322944, 0, 0.5291503667831421, 0, 0],
 [0.7353353798389435, 0, 0, 0.3155672922730446, 0, 0, 0],
 [0,
  0.3503738343715668,
  0.9473413228988647,
  0.2772431564517319,
  0.2051386684179306,
  0.3008444905281067,
  0],
 [0, 0, 0.9513964653015136, 0.3079771399497986, 0.3163562417030334, 0, 0],
 [0, 0.9999984502792358, 0, 0, 0.1739282757043838, 0, 0],
 [0,
  0.7475153803825378,
  0,
  0,
  0.8519415855407715,
  0.5209540128707886,
  0.9566513895988464],
 [0, 0, 0.404523104429245, 0.0895131975412368, 0.2296401411294937, 0, 0],
 [0, 0.6387652605772018, 0.5360426306724548, 0.5632560551166534, 0, 0, 0],
 [0, 0.39628566801548004, 0, 0, 0, 0, 0.976010799407959],
 [0, 0.22181129455566406, 0.38981058448553085, 0.5897918492555618, 0, 0, 0],
 [0.6972024738788605,
  0.9975630044937134,
  0.2032715231180191,
  0.9474552869796752,
  0,
  0,
  0.9050750732421876],
 [0, 0, 0, 0.6985581815242767, 0, 0, 0],
 [0.9976856708526612, 0.6871193647384644, 0, 0.39468874658147496, 0, 0, 0],
 [0.9470542669296265, 0.8392125248908997, 0.983055830001831, 0, 0, 0, 0],
 [0, 0.1830735206604004, 0, 0, 0, 0, 0.5817651748657227],
 [0, 0.1763667315244674, 0, 0.39093559887260193, 0, 0, 0],
 [0, 0.9086764752864838, 0, 0.6986735016107559, 0, 0, 0],
 [0, 0.9401506781578064, 0, 0.5750368535518646, 0, 0, 0],
 [0, 0, 0.15843986868858337, 0, 0.447942520181338, 0, 0],
 [0, 0.677143007516861, 0, 0.3245876729488373, 0, 0, 0],
 [0,
  0.580712248881658,
  0.4119803011417389,
  0.16173540242016313,
  0.5180255174636841,
  0,
  0],
 [0, 0, 0.0722702071070671, 0.6277021765708923, 0.5686312913894653, 0, 0],
 [0, 0.3680701442062855, 0.7937846183776855, 0, 0, 0, 0],
 [0, 0, 0, 0.1726616670687993, 0.6385231614112854, 0.7106927037239075, 0],
 [0,
  0.6990181356668472,
  0.986528754234314,
  0.9908964335918427,
  0,
  0.280539333820343,
  0],
 [0, 0.712792177995046, 0, 0.3012395873665809, 0, 0, 0],
 [0, 0.3782924506813288, 0, 0, 0, 0, 0],
 [0.9637864232063292, 0.5199175501863161, 0.8809828758239746, 0, 0, 0, 0],
 [0.727125883102417, 0.2432291656732559, 0, 0, 0.4630696177482605, 0, 0],
 [0, 0, 0.934254825115204, 0, 0, 0, 0],
 [0.9914891719818116, 0.0865883007645607, 0, 0.2399685382843017, 0, 0, 0],
 [0.8761524558067322, 0, 0.2047822922468185, 0.19045627117156982, 0, 0, 0],
 [0, 0.47995344921946526, 0, 0, 0, 0, 0],
 [0, 0.5836714108784994, 0, 0.3922363929450512, 0, 0, 0],
 [0.8075020909309387, 0.7730647325515747, 0, 0.968628227710724, 0, 0, 0],
 [0.9311657547950744, 0.943433403968811, 0, 0.5910688415169716, 0, 0, 0],
 [0, 0, 0.6156893670558929, 0, 0, 0, 0],
 [0.5492483377456665, 0.3685123473405838, 0, 0.975142776966095, 0, 0, 0],
 [0, 0.22468116879463193, 0.3357025384902954, 0, 0.46190613601356745, 0, 0],
 [0.2103320062160492, 0.510488490263621, 0, 0, 0.7076171040534973, 0, 0],
 [0, 0.7530733346939087, 0, 0, 0.5821109190583229, 0, 0],
 [0.9887515902519226,
  0.8558133244514465,
  0,
  0,
  0.4675808846950531,
  0,
  0.895248293876648],
 [0, 0.5724770203232765, 0, 0.9999996423721312, 0, 0, 0],
 [0,
  0.1109719574451446,
  0.5588555335998535,
  0,
  0.6349553018808365,
  0.4018196165561676,
  0.2173197120428085],
 [0, 0.0444766990840435, 0, 0.2880449593067169, 0, 0, 0.755434513092041],
 [0.9753471612930298,
  0.1466142833232879,
  0,
  0.5031121894717216,
  0,
  0,
  0.37667442858219147],
 [0.68071049451828, 0.821828305721283, 0, 0.13692355714738363, 0, 0, 0],
 [0, 0.3579537719488144, 0.6637248024344444, 0.9874749779701232, 0, 0, 0],
 [0.8901060819625854, 0.15625682473182673, 0.2010165303945541, 0, 0, 0, 0],
 [0, 0.7510431607564291, 0, 0.9547904133796692, 0, 0, 0],
 [0.8836984038352966, 0.6796407699584961, 0, 0, 0, 0, 0],
 [0, 0.44729113578796387, 0, 0.6733046770095825, 0, 0, 0],
 [0.975016951560974, 0.1876919567584991, 0, 0.5496038943529129, 0, 0, 0],
 [0, 0.3202250227332115, 0, 0.571772076189518, 0.1467051804065704, 0, 0],
 [0.4290705323219299,
  0,
  0.8583927750587463,
  0,
  0.1655242592096328,
  0,
  0.8862242698669434],
 [0, 0.0584130361676216, 0, 0.6802905891090631, 0, 0.4347957372665405, 0],
 [0, 0, 0.384603351354599, 0.349162923792998, 0, 0, 0],
 [0.8730899691581726,
  0.1734088957309723,
  0,
  0.5663340166211128,
  0.3764902502298355,
  0,
  0],
 [0,
  0.9222001135349274,
  0.3234823644161224,
  0.4281940460205078,
  0.9070896506309508,
  0,
  0],
 [0, 0, 0, 0.4410122036933899, 0, 0, 0],
 [0.8346452713012695,
  0.1967104971408844,
  0,
  0.2859788089990616,
  0.9323501586914062,
  0,
  0],
 [0.8690880338350931, 0, 0, 0, 0, 0.0839934423565864, 0],
 [0, 0, 0, 0.4174357950687408, 0.8870745301246643, 0, 0],
 [0, 0.1058510914444923, 0, 0.2047339826822281, 0.7394542098045349, 0, 0],
 [0,
  0.6210158169269562,
  0.1737950146198272,
  0.23094109197457627,
  0.9999322891235352,
  0,
  0],
 [0.8784289360046387,
  0.3039485563834508,
  0,
  0.2875932529568672,
  0,
  0,
  0.7574154138565063],
 [0, 0, 0, 0.28843718270460766, 0.7088857889175415, 0, 0],
 [0, 0.5631498346726099, 0, 0, 0.515790268778801, 0, 0],
 [0, 0.6232825070619583, 0, 0.5245604068040848, 0.9671982526779176, 0, 0],
 [0.9419534802436828,
  0.1688170880079269,
  0,
  0.3775460471709569,
  0,
  0,
  0.1915683895349502],
 [0.9919490814208984, 0.2060367614030838, 0, 0, 0, 0, 0],
 [0, 0.5886640052000681, 0, 0.7947491010030111, 0, 0, 0],
 [0, 0.7619930803775787, 0, 0.15058701485395432, 0.5990972518920898, 0, 0],
 [0, 0, 0, 0.28868946433067316, 0, 0, 0],
 [0.999836564064026, 0.9977853894233704, 0, 0.1932415962219238, 0, 0, 0],
 [0.9725577533245087, 0, 0, 0, 0, 0, 0],
 [0.7887290716171265,
  0.9011455178260804,
  0,
  0.5838282306989034,
  0,
  0,
  0.2373962700366974],
 [0, 0.8586428761482239, 0, 0, 0.982642650604248, 0, 0],
 [0, 0, 0, 0.4405561722815037, 0, 0, 0],
 [0, 0.701813280582428, 0, 0.5304955095052718, 0, 0, 0],
 [0, 0.572505310177803, 0, 0.1346357762813568, 0, 0, 0],
 [0.855043351650238,
  0.9763086438179016,
  0.8073421120643616,
  0.26600638404488564,
  0.8395379781723022,
  0,
  0],
 [0, 0.5962350964546204, 0.8225999176502228, 0, 0, 0, 0],
 [0.58710777759552,
  0.6414648294448853,
  0,
  0.4321666806936264,
  0.8989986181259155,
  0,
  0],
 [0, 0, 0, 0.5618552193045616, 0.9931278228759766, 0, 0],
 [0.852134644985199, 0.193113163113594, 0, 0.1132744774222374, 0, 0, 0],
 [0.8500884175300598,
  0.35060690840085346,
  0.964422345161438,
  0,
  0.7344199419021606,
  0,
  0],
 [0, 0.7404531836509705, 0, 0.7889294475317001, 0, 0, 0],
 [0.8631029923756918, 0.3579963967204094, 0, 0, 0, 0, 0],
 [0.5425298810005188,
  0.42797674238681793,
  0,
  0.5212516188621521,
  0.2561501860618591,
  0,
  0],
 [0, 0.8806353211402893, 0, 0.8989295164744059, 0, 0, 0],
 [0, 0.7442100048065186, 0, 0.29551886518796283, 0.5291455388069153, 0, 0],
 [0.9933944940567017,
  0.5130248218774796,
  0.0070564872585237,
  0.6388458013534546,
  0.5870093703269958,
  0,
  0.998559296131134],
 [0.4586570560932159,
  0.5797359123826026,
  0,
  0.7065043449401855,
  0,
  0.9606325030326844,
  0],
 [0, 0, 0, 0.0298800654709339, 0, 0, 0],
 [0.9985498785972596,
  0.8576722741127014,
  0.2047391384840011,
  0,
  0.5726414620876312,
  0,
  0],
 [0,
  0.2717212438583374,
  0.7099677920341492,
  0.7404987514019012,
  0.9853609204292296,
  0,
  0],
 [0,
  0.38433055579662323,
  0.9606517553329468,
  0.1980326175689697,
  0.0018733113538473,
  0,
  0],
 [0.4756860733032226, 0.9405284523963928, 0, 0.967712938785553, 0, 0, 0],
 [0,
  0.9996680021286012,
  0,
  0.8830722570419312,
  0.6615848541259766,
  0.7329308390617371,
  0],
 [0, 0.2166718542575836, 0, 0.5439864993095398, 0, 0, 0],
 [0.7422663569450378, 0.21673345565795896, 0.7736464738845825, 0, 0, 0, 0],
 [0.9613361954689026, 0, 0, 0.0967212840914726, 0, 0.2987103462219238, 0],
 [0, 0.1627363413572311, 0.22793797403573984, 0, 0.1728652417659759, 0, 0],
 [0.85548335313797,
  0.9657625555992126,
  0.0875521004199981,
  0.8676820000012716,
  0,
  0,
  0],
 [0.7478852272033691,
  0.40291969776153563,
  0.7863990068435669,
  0.4353816509246826,
  0,
  0,
  0],
 [0.7881165146827698, 0.1376917064189911, 0, 0, 0, 0, 0],
 [0.5237802565097809, 0, 0, 0.1531589776277542, 0, 0, 0],
 [0.9227574467658995, 0.6388328075408936, 0, 0.9583064913749696, 0, 0, 0],
 [0,
  0.4851929247379303,
  0.5078431963920593,
  0.996380627155304,
  0.9973886609077454,
  0,
  0],
 [0.7543229758739471, 0.4663476049900055, 0.8870601058006287, 0, 0, 0, 0],
 [0.8350342512130737,
  0.6160757094621658,
  0.47502748668193817,
  0,
  0.4402572214603424,
  0,
  0],
 [0,
  0.24915181100368497,
  0.939636766910553,
  0.1481101214885711,
  0.48844029754400253,
  0,
  0],
 [0.7550103068351746, 0, 0, 0.7853804230690002, 0.8235424757003784, 0, 0],
 [0, 0, 0, 0.5348197221755981, 0, 0, 0],
 [0.1941781342029571, 0, 0.8209009766578674, 0.2677595913410187, 0, 0, 0],
 [0.806919515132904, 0, 0.0683556199073791, 0, 0, 0, 0],
 [0, 0.185909554362297, 0, 0.8055852850278219, 0, 0, 0],
 [0.9983119964599608, 0.9582659006118774, 0, 0.5786018744111061, 0, 0, 0],
 [0, 0.6757785946130752, 0.6325847506523132, 0.1677526533603668, 0, 0, 0],
 [0, 0.2769373804330826, 0, 0, 0, 0, 0.1732809245586395],
 [0.9066005746523539, 0, 0, 0, 0, 0, 0],
 [0, 0, 0.2032179534435272, 0.7523135393857956, 0.9789738655090332, 0, 0],
 [0, 0, 0.1818059906363487, 0, 0.8074188530445099, 0, 0],
 [0, 0.16265640258789055, 0.41434724628925323, 0, 0, 0, 0],
 [0, 0.41386882308870554, 0, 0.2464707543452581, 0, 0, 0],
 [0, 0.941849946975708, 0, 0.9994484782218932, 0, 0, 0.6391354203224182],
 [0, 0.2578998804092407, 0, 0.23977633193135262, 0.8098289966583252, 0, 0],
 [0.9919453859329224, 0, 0, 0.5864589810371399, 0, 0, 0],
 [0, 0.2194991111755371, 0, 0.5761193633079529, 0.9166260957717896, 0, 0],
 [0, 0, 0, 0.9899231791496276, 0, 0, 0],
 [0, 0, 0, 0.13992113806307316, 0, 0, 0],
 [0, 0.9023181398709615, 0, 0.9544405043125153, 0.8746390342712402, 0, 0],
 [0,
  0.5993314683437347,
  0.2393578290939331,
  0.4413269162178039,
  0,
  0.3032544255256653,
  0],
 [0, 0.5587488412857056, 0.327093094587326, 0.7474697828292847, 0, 0, 0],
 [0.9542343020439148,
  0.8732752501964569,
  0.7260780930519104,
  0,
  0.7888891696929932,
  0,
  0],
 [0.9930824041366576, 0.5781004726886749, 0, 0.19251259416341782, 0, 0, 0],
 [0, 0, 0.6722255945205688, 0.48293141027291614, 0, 0.0661607608199119, 0],
 [0.9962896307309469, 0.6108551422754923, 0, 0, 0, 0, 0],
 [0, 0.5478115528821945, 0, 0, 0.2585894465446472, 0.7577463388442993, 0],
 [0, 0.5244430899620056, 0, 0, 0, 0.7778548002243042, 0],
 [0, 0.9031608700752258, 0, 0.6722092827161154, 0, 0, 0],
 [0, 0.5359324067831039, 0.9795721173286438, 0, 0, 0, 0],
 [0, 0.9085646867752076, 0, 0.5590359196066856, 0.5553073063492775, 0, 0],
 [0, 0, 0.3049352864424388, 0.4335543115933736, 0, 0, 0],
 [0.9655671715736388, 0.187717318534851, 0, 0.8277181386947632, 0, 0, 0],
 [0.4928815364837646,
  0.1938001712163289,
  0.0878019407391548,
  0,
  0.8594896793365479,
  0.0833527296781539,
  0],
 [0, 0, 0, 0.7190126180648804, 0, 0, 0],
 [0, 0.693662424882253, 0, 0, 0, 0, 0],
 [0.9396730065345764, 0.6671939492225647, 0, 0.4351479560136795, 0, 0, 0],
 [0.8580000400543213,
  0.3792301416397095,
  0,
  0.1384925171732902,
  0,
  0.762707531452179,
  0],
 [0, 0.3337550163269043, 0, 0.8392081409692764, 0.8199997544288635, 0, 0],
 [0.2769265472888946, 0.4647690951824188, 0, 0.8350702524185181, 0, 0, 0],
 [0,
  0.4238203763961792,
  0.4119803011417389,
  0.12511056195944545,
  0.7204829454421997,
  0,
  0],
 [0.2009294182062149,
  0.1401687413454055,
  0.1341946125030517,
  0.33011963218450546,
  0,
  0,
  0],
 [0.9856629967689514, 0.5685001760721207, 0, 0, 0, 0, 0],
 [0.965239405632019, 0, 0.864943265914917, 0.24612613767385477, 0, 0, 0],
 [0, 0.4858341217041015, 0, 0.2506068646907806, 0.7429352402687073, 0, 0],
 [0,
  0.5707161575555801,
  0,
  0.7039675116539001,
  0.9670658707618712,
  0.7457398772239685,
  0],
 [0, 0, 0, 0.2697717547416687, 0, 0, 0],
 [0, 0, 0.4910050332546234, 0, 0.7110612988471985, 0, 0],
 [0.6488620638847351, 0.685554027557373, 0, 0.5151886940002441, 0, 0, 0],
 [0, 0, 0.0732994824647903, 0.1869460195302963, 0, 0, 0],
 [0, 0.2229778170585632, 0, 0, 0.6201033592224121, 0.2956096231937408, 0],
 [0, 0, 0, 0, 0.37567200511693954, 0, 0],
 [0, 0.9746620655059814, 0.1282423138618469, 0, 0.7275846004486084, 0, 0],
 [0, 0.5618903736273447, 0, 0.2859173417091369, 0, 0, 0],
 [0, 0, 0, 0.736616313457489, 0, 0, 0],
 [0, 0.7650484442710876, 0, 0, 0, 0, 0],
 [0, 0.44517527520656586, 0, 0.3837544918060303, 0, 0, 0],
 [0, 0.2042194902896881, 0, 0.9999996423721312, 0, 0, 0],
 [0.8275470435619354,
  0.8329088091850281,
  0,
  0,
  0.7790562212467194,
  0.0180742442607879,
  0.2330736517906189],
 [0, 0.1650880724191665, 0, 0, 0.4077780246734619, 0, 0],
 [0, 0.9469757080078124, 0.39196640253067017, 0, 0.3988919258117676, 0, 0],
 [0, 0.3864554166793823, 0.8111147284507751, 0.3964209146797657, 0, 0, 0],
 [0.7305022925138474,
  0.3911655694246292,
  0,
  0.2145054787397384,
  0.7364519238471985,
  0.494967371225357,
  0],
 [0, 0, 0.7642458081245422, 0.6462923914194107, 0, 0, 0],
 [0.9435834884643555,
  0.6614264696836472,
  0.6317604680856069,
  0.2246100455522537,
  0,
  0,
  0],
 [0.8157311677932739, 0, 0, 0.7235521078109741, 0.9664013385772704, 0, 0],
 [0, 0, 0.3055712878704071, 0, 0.6060523986816406, 0, 0],
 [0.8975757360458374, 0.24436493590474126, 0, 0.5727054923772812, 0, 0, 0],
 [0, 0, 0, 0.1169563345611095, 0, 0, 0],
 [0, 0.2989444211125374, 0, 0.24394591152667994, 0, 0, 0],
 [0.9988512992858888, 0, 0, 0.9636210203170776, 0, 0, 0],
 [0, 0.9999988873799642, 0, 0, 0, 0, 0],
 [0, 0, 0, 0.3065113474925359, 0, 0, 0],
 [0.9993160963058472, 0, 0, 0, 0, 0, 0],
 [0, 0, 0, 0.1874303966760635, 0.490555852651596, 0, 0],
 [0, 0, 0, 0.5091973543167114, 0.4614312648773193, 0, 0],
 [0.7321107983589172, 0.4378501698374748, 0, 0, 0, 0, 0],
 [0.97362220287323, 0.24818605184555054, 0, 0, 0, 0, 0],
 [0, 0.1812153309583664, 0, 0.658309648434321, 0, 0, 0.2665957510471344],
 [0.335129827260971, 0.6240821182727814, 0.5882979109883308, 0, 0, 0, 0],
 [0,
  0.8018253445625305,
  0,
  0.8372848033905029,
  0.9061606228351593,
  0.0847030505537986,
  0],
 [0, 0.7201951742172241, 0.9261971414089203, 0, 0.9654997885227203, 0, 0],
 [0, 0.8262170553207397, 0, 0.7899132072925568, 0, 0, 0],
 [0, 0.619205892086029, 0, 0.7338870763778687, 0.7658072113990784, 0, 0],
 [0, 0, 0.8282636404037476, 0.4825395345687866, 0, 0, 0],
 [0, 0, 0, 0.929259955883026, 0, 0, 0],
 [0, 0, 0.2409137934446334, 0.1509837955236435, 0, 0, 0],
 [0.4535481929779053, 0, 0.2210901826620102, 0.4547971189022064, 0, 0, 0],
 [0.6479278405507406, 0, 0.3820513486862182, 0, 0.8259788155555725, 0, 0],
 [0, 0.7567852834860483, 0, 0.7087427973747253, 0, 0, 0],
 [0.997560739517212, 0, 0, 0.6582297086715698, 0.5875887274742126, 0, 0],
 [0, 0, 0, 0.7728211085001627, 0, 0, 0],
 [0, 0.2485635578632354, 0.2658907771110534, 0.1572520732879638, 0, 0, 0],
 [0, 0.5149025470018387, 0, 0.9478759169578552, 0, 0, 0],
 [0.9961187839508056, 0, 0, 0, 0.2344469502568245, 0, 0],
 [0.9705955386161804, 0.955918788909912, 0, 0, 0.9029898047447203, 0, 0],
 [0, 0, 0, 0.6775863369305929, 0, 0, 0],
 [0, 0.7591628432273865, 0, 0.127221331000328, 0, 0, 0],
 [0.7110794186592102,
  0.680536150932312,
  0,
  0.7004865407943726,
  0.1321019381284713,
  0,
  0.5583259463310242],
 [0,
  0.771450936794281,
  0.9231293201446532,
  0.0858028084039688,
  0,
  0,
  0.9479941725730896],
 [0, 0.5506677590310574, 0, 0, 0.6888405680656433, 0, 0],
 [0.6529971659183502,
  0.937824785709381,
  0,
  0.0750802159309387,
  0.2313665226101875,
  0,
  0],
 [0.9834801554679872, 0, 0, 0.6326945200562477, 0, 0, 0],
 [0, 0.5678587555885315, 0, 0.17532622814178467, 0.484655037522316, 0, 0],
 [0, 0, 0, 0.32703647017478943, 0, 0, 0],
 [0, 0, 0, 0.2737424820661545, 0, 0, 0],
 [0, 0.937646746635437, 0, 0.8837233781814575, 0, 0, 0],
 [0, 0, 0.9485570788383484, 0, 0.1020479425787925, 0, 0],
 [0, 0.35078610479831696, 0, 0.5737656131386757, 0, 0, 0.1929570734500885],
 [0, 0, 0, 0.6185382902622223, 0, 0, 0],
 [0.9589570462703706, 0, 0, 0.7784577012062073, 0.6921831369400024, 0, 0],
 [0, 0.46299326717853545, 0, 0.1336478590965271, 0, 0, 0],
 [0.8685146421194077, 0.7723398605982462, 0, 0, 0.75841224193573, 0, 0],
 [0, 0, 0.8820732831954956, 0.461660772562027, 0, 0, 0],
 [0, 0.1851717382669448, 0, 1.0, 0.9781557321548462, 0, 0.7830358147621155],
 [0.9826825857162476, 0.9704091548919678, 0, 0.563935711979866, 0, 0, 0],
 [0, 0.0974400043487548, 0, 0, 0.5456193089485168, 0, 0],
 [0, 0.1933234147727489, 0, 0.9050100445747375, 0, 0, 0],
 [0, 0.7483464876810709, 0.8457674980163574, 0.7465699315071106, 0, 0, 0],
 [0.9038006365299225,
  0.3328661918640136,
  0,
  0.5911608934402466,
  0.4549315050244331,
  0,
  0],
 [0, 0.7407014966011047, 0, 0.5732750296592711, 0, 0, 0],
 [0, 0.4474040796048939, 0, 0.1447875276207924, 0, 0, 0],
 [0, 0.4382002651691437, 0, 0.2063779979944229, 0, 0, 0],
 [0, 0.4361404553055763, 0, 0, 0, 0, 0],
 [0, 0.5319429636001587, 0, 0, 0.5433284938335419, 0, 0],
 [0.9301759004592896,
  0.9433960914611816,
  0.3798804084459941,
  0,
  0.1455951482057571,
  0,
  0],
 [0, 0.2643985450267792, 0.1730610579252243, 0, 0, 0, 0],
 [0, 0, 0, 0.9145642638206481, 0, 0, 0],
 [0.9985268115997314,
  0.6000422785679499,
  0,
  0.996598184108734,
  0,
  0.0335448645055294,
  0],
 [0, 0.4569946179787318, 0.5880762040615082, 0, 0, 0, 0],
 [0, 0.8935765624046326, 0, 0, 0, 0, 0],
 [0, 0.8382226228713989, 0.9002190828323364, 0.9824780821800232, 0, 0, 0],
 [0.9947651624679564,
  0,
  0.3537010550498962,
  0.2705543637275696,
  0,
  0.7972285151481628,
  0],
 [0.919966995716095, 0.4640173465013504, 0, 0, 0, 0, 0],
 [0, 0, 0, 0.9810948371887208, 0, 0, 0],
 [0, 0.5346655771136284, 0, 0.1866682320833206, 0, 0, 0],
 [0, 0.1857891976833343, 0, 0.9999996423721312, 0, 0, 0],
 [0.9850815534591676, 0, 0, 0, 0, 0, 0],
 [0, 0.1264423262327909, 0.2143518179655075, 0.1607735604047775, 0, 0, 0],
 [0, 0.6305590867996216, 0, 0.7460231900215148, 0, 0.9326297640800476, 0],
 [0.7780826687812805, 0.5265422612428665, 0, 0, 0, 0, 0.9470831155776978],
 [0.9963617523511251, 0, 0, 0.9729297161102295, 0, 0, 0],
 [0,
  0.818442165851593,
  0.0355157554149627,
  0,
  0.9288597106933594,
  0,
  0.9875763654708862],
 [0, 0.32258214553197223, 0, 0, 0.3297072052955627, 0, 0],
 [0, 0.1463252902030944, 0, 0.3490307629108429, 0.8519936800003052, 0, 0],
 [0.5140544846653938, 0.2713688015937805, 0, 0, 0.5669681429862976, 0, 0],
 [0.2421191185712814,
  0.9376945793628693,
  0.9879288077354432,
  0.22031734511256212,
  0,
  0,
  0],
 [0, 0.2101299613714218, 0, 0, 0, 0, 0],
 [0, 0, 0.9926453828811646, 0.3753054738044739, 0, 0, 0],
 [0.9788824319839478, 0, 0, 0, 0, 0, 0],
 [0, 0.4360128790140152, 0.0786635503172874, 0, 0.8131541013717651, 0, 0],
 [0, 0.6673983931541443, 0, 0.6826788783073425, 0.9965460896492004, 0, 0],
 [0.7375098764896393, 0.17530412971973414, 0, 0, 0, 0.1116907000541687, 0],
 [0, 0.6798208951950073, 0, 0.9186642169952393, 0.882199764251709, 0, 0],
 [0, 0, 0, 0, 0.9744198322296144, 0, 0],
 [0, 0.5180510655045509, 0, 0.5271496176719666, 0, 0.1489452272653579, 0],
 [0, 0.24346860249837235, 0, 0.0954240560531616, 0, 0, 0],
 [0.5937815308570862,
  0,
  0.4605516791343689,
  0.368699312210083,
  0,
  0.40830525755882263,
  0],
 [0, 0.2796841561794281, 0, 0, 0.4911365807056427, 0, 0],
 [0.7459213733673096, 0.2256866991519928, 0, 0, 0, 0, 0],
 [0.213696002960205, 0.4192092716693878, 0, 0.0761165618896484, 0, 0, 0],
 [0, 0.7398661375045776, 0, 0.551514595746994, 0.5393770933151245, 0, 0],
 [0, 0, 0.2122107595205307, 0.1820798814296722, 0, 0, 0],
 [0, 0.4836683012545109, 0, 0.7505519688129425, 0, 0, 0],
 [0.9999179840087892, 0.5999712273478508, 0, 0, 0, 0, 0],
 [0.8032838503519694, 0, 0.4345232844352722, 0, 0.0651611536741256, 0, 0],
 [0, 0.4768611093362172, 0, 0.1991864169637362, 0.9044187664985656, 0, 0],
 [0, 0, 0, 0.21670018136501304, 0.5198571979999542, 0, 0],
 [0.8426806628704071, 0.6201927661895752, 0, 0, 0, 0, 0],
 [0, 0.2593645751476288, 0, 0.9821004271507264, 0, 0, 0],
 [0, 0.982747495174408, 0.9602136611938475, 0, 0.9433811902999878, 0, 0],
 [0.9076706767082214, 0, 0, 0.66035957634449, 0.7630181908607483, 0, 0],
 [0, 0.9998528957366944, 0, 0, 0, 0, 0],
 [0, 0.4077863196531932, 0, 0, 0, 0, 0],
 [0.7787826458613077, 0.917360246181488, 0, 0, 0.9904737869898478, 0, 0],
 [0, 0, 0.1877351701259613, 0, 0.3107549250125885, 0, 0],
 [0, 0, 0, 0.2439068257808685, 0, 0, 0],
 [0.6602874398231506, 0.8078694343566895, 0, 0, 0, 0, 0],
 [0, 0, 0.2050797790288925, 0, 0.3017663657665252, 0, 0],
 [0, 0.1731486320495605, 0, 0.40965812280774117, 0, 0, 0],
 [0, 0.7061854004859924, 0, 0.4897054433822632, 0, 0, 0],
 [0, 0, 0, 0, 0.6699798107147217, 0, 0.037310604006052],
 [0,
  0.2862441688776016,
  0.3595884405076504,
  0.673895001411438,
  0.965898334980011,
  0,
  0],
 [0, 0, 0.6258012652397156, 0, 0, 0, 0],
 [0, 0.962145745754242, 0, 0, 0.2287657409906387, 0, 0],
 [0.8609865307807922, 0, 0, 0, 0, 0, 0],
 [0.8553982973098755, 0, 0, 0, 0.3633233606815338, 0, 0],
 [0.2636506855487823,
  0.6212840874989828,
  0,
  0.9239733219146729,
  0.8916575312614441,
  0,
  0],
 [0, 0.38421063125133514, 0, 0.17695020760099087, 0, 0, 0],
 [0.9912809729576112,
  0.9999048709869384,
  0,
  0.9207552075386047,
  0.9940647482872008,
  0,
  0],
 [0, 0.9270152449607848, 0, 0.1730960756540298, 0, 0, 0.8004189729690552],
 [0, 0.3977418579161167, 0, 0.5212080975373586, 0.2690344154834747, 0, 0],
 [0.8221134543418884,
  0.7325722724199295,
  0.6481255739927291,
  0.1682657152414322,
  0,
  0,
  0],
 [0, 0.9971451163291932, 0, 0, 0, 0, 0],
 [0, 0.3133101001381874, 0, 0.9310728311538696, 0, 0, 0],
 [0, 0, 0, 0.3382214456796646, 0, 0.6580610871315002, 0],
 [0, 0.5439648628234863, 0, 0.34829632317026454, 0, 0, 0],
 [0, 0, 0, 0.922444224357605, 0, 0, 0],
 [0.50998904556036, 0.3779731690883636, 0, 0, 0, 0, 0],
 [0,
  0.9999864101409912,
  0,
  0.9614586234092712,
  0.9924091100692748,
  0,
  0.9981535077095032],
 [0, 0.577920138835907, 0, 0.9898788928985596, 0.9836265742778778, 0, 0],
 [0.783454030752182,
  0.6939151138067245,
  0.7898792624473572,
  0.24179781973361963,
  0,
  0,
  0],
 [0, 0.4286711911360423, 0, 0.605860710144043, 0, 0, 0],
 [0.7800664901733398,
  0.1628996878862381,
  0.4564109109342098,
  0,
  0.3838036656379699,
  0,
  0],
 [0, 0.6790453506012758, 0, 0.9905319213867188, 0, 0, 0],
 [0, 0, 0.3946719169616699, 0.6178484658400217, 0, 0, 0],
 [0, 0.8250210881233215, 0, 0, 0.7050862312316895, 0, 0],
 [0, 0.5433710813522339, 0.0554536581039428, 0.535403236746788, 0, 0, 0],
 [0, 0.9568869471549988, 0.8602281212806702, 0.7569817304611206, 0, 0, 0],
 [0, 0, 0.2121429592370987, 0.0983974933624267, 0.9478491544723512, 0, 0],
 [0.9482046365737916,
  0.0600416213274002,
  0.0878019407391548,
  0,
  0.6191686689853668,
  0,
  0],
 [0,
  0.2743957017858823,
  0,
  0.5266064405441284,
  0.6893755197525024,
  0,
  0.0138356192037463],
 [0, 0.1012224182486534, 0.9265624284744264, 0, 0, 0, 0],
 [0.9762431979179382, 0, 0, 0.32179225981235504, 0, 0, 0],
 [0,
  0.6158140500386556,
  0.8969684839248657,
  0.6303912252187729,
  0.8162943720817566,
  0,
  0],
 [0.9346108436584472, 0, 0, 0.7735064625740051, 0, 0, 0],
 [0.8939879536628723, 0.3849499747157097, 0, 0, 0.1922998875379562, 0, 0],
 [0, 0.48000309616327286, 0, 0.9997653365135192, 0, 0, 0.3053556382656097],
 [0, 0.939242959022522, 0, 0, 0, 0, 0],
 [0, 0.4210429787635803, 0.5537383556365967, 0.6818510293960571, 0, 0, 0],
 [0, 0.05564752686768765, 0, 0, 0, 0, 0],
 [0, 0.24095069617033, 0, 0.3036436438560486, 0, 0, 0.1075439527630806],
 [0.7123711705207825,
  0.2285552620887756,
  0.2530393600463867,
  0.0988438427448272,
  0,
  0,
  0],
 [0.9999996423721312, 0.997621476650238, 0, 0.9636498987674713, 0, 0, 0],
 [0.6475989073514938,
  0.4284641444683075,
  0,
  0.8478533029556274,
  0.9540753364562988,
  0,
  0],
 [0, 0, 0, 0.3189812898635864, 0, 0, 0],
 [0, 0.5986007899045944, 0, 0.5123546347022057, 0.2957467138767242, 0, 0],
 [0, 0.6675703525543213, 0, 0, 0.3049703538417816, 0, 0],
 [0,
  0.9224998354911804,
  0,
  0.6399284303188324,
  0.3464513123035431,
  0,
  0.7380481362342834],
 [0, 0, 0, 0.1701306104660034, 0, 0, 0],
 [0, 0, 0, 0, 0.970702052116394, 0, 0.0840241238474845],
 [0, 0.2939529170592626, 0.9781488180160522, 0, 0.8384366631507874, 0, 0],
 [0,
  0.3768292069435119,
  0.8774186968803406,
  0.467398464679718,
  0,
  0.9011183381080629,
  0],
 [0, 0.7990977168083191, 0, 0.5446871519088745, 0.5082175191491842, 0, 0],
 [0, 0.1095320880413055, 0, 0, 0, 0, 0],
 [0, 0.3572186181942622, 0, 0, 0, 0, 0],
 [0.9888333082199096, 0, 0.7707316875457764, 0, 0, 0, 0],
 [0, 0, 0, 0.9135978519916534, 0, 0, 0],
 [0.9999566078186036, 0, 0, 0.7479395270347595, 0, 0, 0],
 [0, 0.4713231921195984, 0, 0.7501673102378845, 0, 0, 0],
 [0, 0.8097033898035685, 0, 0, 0.7898019552230835, 0, 0],
 [0.7853492498397827,
  0.2264062613248825,
  0.6652280688285828,
  0.2589850127696991,
  0,
  0,
  0],
 [0, 0.1148651763796806, 0, 0, 0, 0.9234976172447203, 0],
 [0, 0.347608208656311, 0, 0.30149710923433304, 0, 0, 0],
 [0, 0.3033813834190368, 0, 0, 0, 0, 0],
 [0.9251611828804016,
  0,
  0.548084999124209,
  0.1676252335309982,
  0.941423237323761,
  0,
  0],
 [0, 0.2315873503684997, 0, 0.0809566527605056, 0.524580717086792, 0, 0],
 [0, 0.3360502123832702, 0.9642099142074584, 0.6644876599311829, 0, 0, 0],
 [0.9838617444038392, 0.5211476534605026, 0, 0, 0, 0, 0],
 [0.7239340543746948, 0.45574740444620443, 0.692572757601738, 0, 0, 0, 0],
 [0, 0, 0.9572142163912455, 0.523177981376648, 0.386613667011261, 0, 0],
 [0, 0.9896531701087952, 0.9529829025268556, 0, 0, 0, 0],
 [0.1780087649822235, 0.5489380608002344, 0.0674691498279571, 0, 0, 0, 0],
 [0.8553030490875244,
  0.8149787485599518,
  0.7120435237884521,
  0.9060190320014954,
  0,
  0,
  0],
 [0, 0, 0, 0.3105745315551758, 0, 0, 0],
 [0.8385533690452576, 0.6635442078113556, 0, 0, 0.695237934589386, 0, 0],
 [0, 0.5707869082689285, 0, 0, 0, 0, 0],
 [0, 0.48639462391535443, 0, 0, 0.2835976481437683, 0, 0],
 [0, 0, 0, 0, 0, 0, 0.089213527739048],
 [0, 0.9669797420501708, 0, 0.1481857299804687, 0, 0, 0],
 [0, 0, 0, 0.0787472650408744, 0, 0, 0],
 [0.9830586910247804, 0.5943448692560196, 0.9935729503631592, 0, 0, 0, 0],
 [0, 0.584276556968689, 0.3518165349960327, 0.9768030643463136, 0, 0, 0],
 [0, 0.8027613162994385, 0, 0.4388434439897537, 0, 0, 0],
 [0,
  0.2356525212526321,
  0.0331190750002861,
  0.0952656045556068,
  0,
  0,
  0.7619056105613708],
 [0, 0, 0, 0.1374349892139434, 0, 0, 0],
 [0, 0.1481355801224708, 0, 0, 0, 0, 0],
 [0, 0.781432569026947, 0, 0.8084046244621277, 0, 0, 0],
 [0, 0.7181117534637451, 0, 0, 0, 0, 0],
 [0.5727640390396118, 0.7277653217315674, 0.4432311058044433, 0, 0, 0, 0],
 [0, 0, 0.3616742491722107, 0, 0.3747145235538482, 0.2872264981269836, 0],
 [0, 0.3782610297203064, 0, 0.854012668132782, 0.4967236816883087, 0, 0],
 [0, 0.19986538589000702, 0, 0.3909452259540558, 0, 0.1013399288058281, 0],
 [0.9928357005119324, 0, 0, 0.7842077016830444, 0.942022740840912, 0, 0],
 [0, 0.1431154645979404, 0, 0.26619162162144977, 0, 0, 0],
 [0, 0.5705551132559776, 0, 0, 0, 0, 0],
 [0, 0.5234284400939941, 0, 0, 0, 0, 0],
 [0.9374096393585204, 0, 0.8265523314476013, 0, 0, 0, 0],
 [0.8333739638328552, 0, 0, 0.0599147714674472, 0, 0, 0],
 [0.9971029162406922, 0, 0, 0.9498622218767802, 0, 0, 0],
 [0, 0, 0, 0.6124492287635803, 0, 0, 0],
 [0, 0, 0, 0.1473552584648132, 0.8214776515960693, 0, 0],
 [0, 0.2070727404206991, 0.13406430929899216, 0, 0, 0, 0],
 [0, 0.9881119132041931, 0, 0.6079228023687998, 0, 0, 0],
 [0, 0.3620156894127528, 0.8291893005371094, 0, 0.30824635922908783, 0, 0],
 [0, 0.5328339437643687, 0.5795112699270248, 0, 0.6723833680152893, 0, 0],
 [0, 0.5736834853887558, 0, 0, 0, 0, 0],
 [0, 0, 0.9607513248920441, 0, 0.7815884947776794, 0, 0],
 [0, 0.2566887885332107, 0.5224303007125854, 0, 0, 0, 0],
 [0, 0.39935678616166115, 0.9554919600486756, 0.3291648328304291, 0, 0, 0],
 [0, 0.2196373119950294, 0, 0, 0, 0, 0],
 [0, 0.46290088444948196, 0, 0, 0, 0, 0.8789078593254089],
 [0, 0, 0, 0.4082956612110138, 0, 0, 0.5008679628372192],
 [0.8626933395862579,
  0.6772453784942627,
  0.8843284845352173,
  0.9481538534164428,
  0.9478491544723512,
  0,
  0],
 [0, 0, 0, 0.662326592952013, 0.9369852542877196, 0, 0],
 [0.6363880038261414, 0, 0, 0, 0, 0, 0],
 [0.7153323292732239, 0.6597811728715895, 0, 0, 0, 0, 0],
 [0, 0.3617240488529205, 0.2684223353862762, 0.8513832688331604, 0, 0, 0],
 [0, 0.8687215447425842, 0, 0.997450053691864, 0, 0, 0],
 [0.9999873638153076, 0, 0, 0.9986761212348938, 0.828154444694519, 0, 0],
 [0.8142076134681702, 0, 0, 0.64506596326828, 0, 0.0192715562880039, 0],
 [0, 0, 0.7062419652938843, 0, 0, 0, 0],
 [0.5043565630912781, 0, 0, 0, 0.8240281343460083, 0, 0],
 [0.4132416844367981, 0, 0, 0, 0, 0, 0],
 [0, 0, 0.202496200799942, 0.2565383613109588, 0, 0.0394061766564846, 0],
 [0, 0, 0, 0.7405649721622467, 0, 0, 0],
 [0, 0.5842433571815491, 0, 0, 0, 0, 0],
 [0, 0.9134953618049622, 0, 0, 0, 0, 0],
 [0, 0.6991828382015228, 0, 0.1673416048288345, 0, 0, 0.3717274069786072],
 [0, 0, 0.0255616270005702, 0, 0, 0, 0],
 [0, 0.28993574902415276, 0, 0.4029674828052521, 0, 0, 0],
 [0.3208548724651336,
  0,
  0.1485220789909362,
  0.5123230367898941,
  0,
  0.4750406146049499,
  0],
 [0, 0.6834388971328735, 0.1843864470720291, 0.36344597488641733, 0, 0, 0],
 [0, 0.5439689755439758, 0, 0, 0.8017814755439758, 0, 0],
 [0, 0.7887302041053772, 0, 0.6101733818650246, 0.9875646233558656, 0, 0],
 [0.1813472509384155,
  0.4900151789188385,
  0,
  0,
  0.8510906100273132,
  0.2888577878475189,
  0],
 [0, 0.3665411472320556, 0, 0, 0.3824392855167389, 0, 0],
 [0.804774820804596, 0.2093791365623474, 0, 0.4012947976589203, 0, 0, 0],
 [0,
  0,
  0.5327671766281128,
  0.1809339076280594,
  0.7181224822998047,
  0.150772538036108,
  0],
 [0, 0, 0, 0.6814729770024618, 0, 0, 0],
 [0.44602896459400654, 0.8979423642158508, 0.0385872460901737, 0, 0, 0, 0],
 [0.8537272810935974,
  0.5059460904449224,
  0,
  0.9962869882583618,
  0.5474837422370911,
  0,
  0],
 [0, 0, 0.9664808511734008, 0, 0.9849373698234558, 0, 0],
 [0, 0.5215648636221885, 0, 0.1134360954165458, 0, 0, 0.1888028234243393],
 [0, 0.6771321818232536, 0.6375916004180908, 0, 0, 0.4533230066299438, 0],
 [0, 0, 0, 0.31572508811950684, 0, 0, 0],
 [0, 0.5892202705144882, 0, 0, 0, 0, 0],
 [0, 0.715386688709259, 0, 0, 0, 0, 0],
 [0, 0.995960533618927, 0.5360426306724548, 0, 0, 0, 0],
 [0, 0.736011803150177, 0.791957676410675, 0.6457101404666901, 0, 0, 0],
 [0.918582558631897, 0, 0.6923561692237854, 0, 0.6384051144123077, 0, 0],
 [0.7924186786015829, 0.46576949767768383, 0, 0.1192408576607704, 0, 0, 0],
 [0, 0, 0, 0.9482183456420898, 0, 0, 0],
 [0, 0, 0, 0.0853188708424568, 0, 0, 0],
 [0, 0, 0.42654329538345337, 0, 0.198809415102005, 0, 0],
 [0, 0.296381413936615, 0, 0, 0, 0, 0],
 [0, 0.8001119196414948, 0, 0.7656692663828532, 0, 0, 0],
 [0, 0, 0, 0.9085761904716492, 0, 0, 0],
 [0.5727640390396118, 0.5180519148707389, 0, 0.6036589443683623, 0, 0, 0],
 [0, 0.2489706575870514, 0, 0, 0, 0, 0],
 [0.7382850050926208, 0.9706627130508424, 0, 0.1722590327262878, 0, 0, 0],
 [0, 0, 0.9484108090400696, 0, 0, 0, 0],
 [0, 0.47814201315244037, 0.3657584190368652, 0, 0, 0.7399091124534607, 0],
 [0, 0.1228229776024818, 0, 0.6974291205406189, 0, 0, 0],
 [0.9998968839645386, 0, 0.7807781100273132, 0, 0, 0, 0],
 [0, 0.34584666043519974, 0.1672731637954712, 0.5999831389635801, 0, 0, 0],
 [0, 0, 0, 0, 0.1218799129128456, 0, 0],
 [0, 0.9999085664749146, 0, 0, 0, 0, 0],
 [0, 0.0870582610368728, 0, 0.3847594857215881, 0, 0, 0],
 [0, 0.8165372610092163, 0, 0, 0, 0, 0],
 [0, 0, 0, 0.0598405748605728, 0, 0, 0],
 [0.8718810081481934, 0, 0, 0, 0, 0, 0],
 [0.3804613351821899, 0.5263410806655884, 0, 0.7472032010555267, 0, 0, 0],
 [0, 0, 0, 0.9951828122138976, 0, 0, 0],
 [0.9918335676193236, 0, 0, 0.5289152264595032, 0, 0, 0],
 [0.3766694664955139, 0.8329246044158936, 0, 0, 0, 0, 0],
 [0, 0, 0, 0.2462428659200668, 0, 0, 0],
 [0,
  0.3003977686166763,
  0.1341946125030517,
  0.3317366242408752,
  0.2069655805826187,
  0,
  0],
 [0, 0.8575235605239868, 0, 0.8620091080665588, 0, 0, 0],
 [0, 0.97879159450531, 0, 0.4058115966618061, 0.587875247001648, 0, 0],
 [0, 0, 0, 0.2798959910869598, 0, 0, 0],
 [0, 0.5114593505859375, 0, 0.5954097136855125, 0, 0, 0.36953024566173553],
 [0, 0.971643328666687, 0, 0.9005259275436401, 0.894963264465332, 0, 0],
 [0, 0.3621469140052795, 0, 0.6304070353507996, 0, 0.1878715008497238, 0],
 [0.9902136921882628, 0.7737021446228027, 0, 0, 0.8860723376274109, 0, 0],
 [0, 0.1853985637426376, 0, 0.8094807267189026, 0.5984287261962891, 0, 0],
 [0, 0.8410638570785522, 0, 0, 0.5742425918579102, 0, 0],
 [0, 0.39448509365320206, 0, 0, 0.9975784420967102, 0, 0],
 [0, 0, 0, 0.0379988029599189, 0, 0.2622433602809906, 0],
 [0, 0, 0.6475211679935455, 0.9948638081550598, 0.8861631155014038, 0, 0],
 [0, 0.3873155762751897, 0, 0.5641135573387146, 0, 0, 0],
 [0, 0, 0, 0.30073313042521477, 0.41062226891517634, 0, 0],
 [0.890795886516571, 0, 0, 0.942528247833252, 0, 0, 0],
 [0, 0, 0, 0.9442080855369568, 0, 0, 0],
 [0, 0.4016607999801636, 0, 0, 0, 0, 0],
 [0, 0.1103745773434639, 0, 0, 0, 0, 0],
 [0, 0, 0.2744899392127991, 0, 0, 0, 0],
 [0, 0, 0, 0.9999927282333374, 0.9982655644416808, 0, 0],
 [0,
  0.6390618085861206,
  0.1381522268056869,
  0.2653797268867492,
  0.3480162918567657,
  0,
  0],
 [0, 0, 0, 0.41743443533778185, 0, 0, 0],
 [0.9059877991676332, 0.999525785446167, 0, 0.3291953913867473, 0, 0, 0],
 [0.846954882144928, 0.9847377836704254, 0, 0.09320053582390143, 0, 0, 0],
 [0.9663292765617372, 0, 0.26960317417979235, 0.5588350892066956, 0, 0, 0],
 [0, 0, 0, 0.23991772780815757, 0, 0, 0],
 [0, 0.2797872424125671, 0, 0, 0, 0, 0],
 [0, 0.4227174073457718, 0, 0.5048622290293375, 0.9865593910217284, 0, 0],
 [0.6334412693977356, 0.4946913868188858, 0, 0, 0.753395676612854, 0, 0],
 [0.942265659570694, 0, 0.1436433941125869, 0, 0, 0, 0],
 [0, 0, 0, 0.2865998148918152, 0.8375911116600037, 0, 0],
 [0, 0.0597660616040229, 0, 0.1155664026737213, 0, 0, 0],
 [0, 0.3219424486160278, 0, 0, 0.8354579210281372, 0, 0],
 [0, 0.933258593082428, 0.4212210476398468, 0, 0, 0, 0],
 [0, 0, 0, 0.18798514641821384, 0, 0, 0],
 [0, 0.5522451996803284, 0, 0.5758950114250183, 0, 0, 0],
 [0.875389575958252, 0.9254366755485536, 0, 0, 0, 0, 0.6699991822242737],
 [0, 0.54494721442461, 0, 0.763841837644577, 0.9464755654335022, 0, 0],
 [0, 0.6046744585037231, 0, 0, 0, 0, 0],
 [0, 0.0170830581337213, 0.2428068369626999, 0, 0.441216766834259, 0, 0],
 [0, 0, 0, 0.5669242739677429, 0, 0.0568880029022693, 0],
 [0, 0.0558788292109966, 0.3875651359558105, 0, 0.2538265436887741, 0, 0],
 [0, 0, 0, 0.6182939410209656, 0.2835976481437683, 0, 0],
 [0, 0.0755827650427818, 0, 0.5474700927734375, 0, 0, 0],
 [0, 0, 0, 0, 0.357115775346756, 0, 0],
 [0, 0, 0.1239846721291542, 0, 0, 0, 0.2588282823562622],
 [0,
  0.7061436176300049,
  0.3597057163715362,
  0.1767493486404419,
  0.6948080658912659,
  0,
  0],
 [0, 0.5949000120162964, 0, 0, 0.367022693157196, 0, 0],
 [0, 0.1204182207584381, 0.4491068422794342, 0, 0.7429479956626892, 0, 0],
 [0, 0.1374718397855758, 0.9613185524940492, 0, 0.6473440527915955, 0, 0],
 [0, 0, 0, 0.8424149751663208, 0.8534520268440247, 0.8857289552688599, 0],
 [0, 0, 0, 0.7495318949222565, 0, 0, 0],
 [0.3275781273841858, 0.2558297514915466, 0, 0, 0, 0, 0],
 [0, 0, 0, 0.4327642172574997, 0.3491645157337188, 0, 0],
 [0, 0.18942046165466303, 0, 0, 0, 0, 0],
 [0, 0.93048095703125, 0, 0, 0, 0, 0],
 [0,
  0.45750362426042557,
  0.1443936675786972,
  0.178844466805458,
  0,
  0,
  0.129309669137001],
 [0, 0.8548882156610489, 0, 0, 0, 0, 0],
 [0, 0.6803147246440252, 0.4102682471275329, 0, 0, 0, 0],
 [0.6531750559806824, 0.5526596307754517, 0, 0, 0.4005244076251983, 0, 0],
 [0, 0.1245761662721633, 0, 0.13963700830936432, 0, 0, 0],
 [0, 0, 0, 0.5437676422297955, 0, 0.9987727999687196, 0],
 [0, 0, 0, 0, 0.9743108749389648, 0, 0],
 [0, 0.8281846841176351, 0, 0.9999954700469972, 0, 0, 0],
 [0.9340490698814392, 0.5792956948280334, 0, 0, 0, 0, 0],
 [0.9999409914016724, 0, 0.7320988178253174, 0.8242762088775635, 0, 0, 0],
 [0.9984747767448424, 0.9791644811630248, 0, 0, 0, 0, 0],
 [0, 0.2230485007166862, 0, 0.2112819850444793, 0, 0, 0],
 [0, 0, 0.8931217789649963, 0, 0.2104215621948242, 0, 0],
 [0, 0.6784105896949768, 0, 0.6258914073308309, 0, 0, 0],
 [0.9975273609161377, 0, 0, 0.5186004638671875, 0, 0, 0],
 [0.581733763217926, 0, 0, 0.8559986352920532, 0, 0, 0],
 [0.3653983771800995, 0.9760369658470154, 0, 0, 0, 0, 0],
 [0, 0, 0.4707983136177063, 0, 0.3716106712818146, 0, 0],
 [0,
  0.5499006360769272,
  0.1369954645633697,
  0.3679050207138061,
  0,
  0.8253938555717468,
  0],
 [0, 0.9348942637443542, 0, 0, 0, 0, 0],
 [0.8243325352668762, 0, 0, 0.9895742535591124, 0, 0, 0],
 [0, 0.9211660623550415, 0.7448468804359436, 0.8644008040428162, 0, 0, 0],
 [0, 0.6340291500091553, 0, 0.1172443553805351, 0, 0, 0],
 ...]
In [21]:
distance_matrix = cdist(workers_df['data'].tolist(), workers_df['data'].tolist(), 'euclidean')
distance_df = pd.DataFrame(distance_matrix, index=workers_df.wid, columns=workers_df.wid)
# g = sns.clustermap(distance_df, center=0, cmap="vlag",
#                    dendrogram_ratio=(.1, .2),
#                    cbar_pos=(.02, .32, .03, .2),
#                    linewidths=.75, figsize=(12, 13))
In [22]:
len(distance_df)
Out[22]:
4644
In [23]:
distance_block_df = distance_df.iloc[:100,:100]
g = sns.clustermap(distance_block_df, center=0, cmap="vlag",
                   dendrogram_ratio=(.1, .2),
                   cbar_pos=(.02, .32, .03, .2),
                   linewidths=.75, figsize=(12, 13))

/Users/chenjianfeng/anaconda3/lib/python3.11/site-packages/seaborn/matrix.py:560: UserWarning: Clustering large matrix with scipy. Installing `fastcluster` may give better performance.
  warnings.warn(msg)
/Users/chenjianfeng/anaconda3/lib/python3.11/site-packages/seaborn/matrix.py:530: ClusterWarning: scipy.cluster: The symmetric non-negative hollow observation matrix looks suspiciously like an uncondensed distance matrix
  linkage = hierarchy.linkage(self.array, method=self.method,
/Users/chenjianfeng/anaconda3/lib/python3.11/site-packages/seaborn/matrix.py:560: UserWarning: Clustering large matrix with scipy. Installing `fastcluster` may give better performance.
  warnings.warn(msg)
/Users/chenjianfeng/anaconda3/lib/python3.11/site-packages/seaborn/matrix.py:530: ClusterWarning: scipy.cluster: The symmetric non-negative hollow observation matrix looks suspiciously like an uncondensed distance matrix
  linkage = hierarchy.linkage(self.array, method=self.method,
In [24]:
len(distance_df.columns)
Out[24]:
4644

3. Further data mining Happiness: How to improve our happiness among marital or single?¶

  • To delve deeper, semantic models are needed, which may require time to select the appropriate one.
  • A simpler and intuitive approach is to analyze word frequency.
In [4]:
#insert the photo
from IPython.display import Image
image_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/figs/happy_third.jpg"
Image(filename=image_path, width=600, height=600)
Out[4]:
In [2]:
#insert the photo
from IPython.display import Image
image_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/figs/workflow.jpg"
Image(filename=image_path, width=600, height=600)
Out[2]:

👷In the third part, progress is being continued. The final version will finished 1-2 days after

🌟The current idea is to use the previous word cloud to infer the scenario, because the third part is mainly to explore "how", so for example, it is explored that for married people, a possible way to improve happiness is for couples to go home together after get off work. Parents pick up and drop off children after school, and single people may get orders from big customers, promotions and salary increases, social recognition, etc. through the workplace.

📒The picture above is the brainstorming flow chart for the third part

In [3]:
## Still Working ...

4. Appendix¶

The reason why marital will improve happiness?¶

In [25]:
import pandas as pd

import matplotlib.pyplot as plt
import seaborn as sns
In [27]:
cleaned_hm_csv_file = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/data/cleaned_hm.csv"
cleaned_hm_df = pd.read_csv(cleaned_hm_csv_file)

demographic_csv_file = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/data/demographic.csv" 
demographic_df = pd.read_csv(demographic_csv_file)
In [28]:
cleaned_hm_df = cleaned_hm_df.dropna(subset=['ground_truth_category'])
In [29]:
cleaned_hm_df.head()
Out[29]:
hmid wid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category
3 27676 206 24h We had a serious talk with some friends of our... We had a serious talk with some friends of our... True 2 bonding bonding
5 27678 45 24h I meditated last night. I meditated last night. True 1 leisure leisure
24 27697 498 24h My grandmother start to walk from the bed afte... My grandmother start to walk from the bed afte... True 1 affection affection
32 27705 5732 24h I picked my daughter up from the airport and w... I picked my daughter up from the airport and w... True 1 bonding affection
42 27715 2272 24h when i received flowers from my best friend when i received flowers from my best friend True 1 bonding bonding
In [30]:
cleaned_hm_df_merge = cleaned_hm_df.merge(demographic_df,on='wid')
In [31]:
cleaned_hm_df_merge.head()
Out[31]:
hmid wid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category age country gender marital parenthood
0 27676 206 24h We had a serious talk with some friends of our... We had a serious talk with some friends of our... True 2 bonding bonding 28 DNK f married n
1 49534 206 24h I bought my wedding shoes! I bought my wedding shoes! True 1 achievement achievement 28 DNK f married n
2 52451 206 24h My new watch arrived in the mail! My new watch arrived in the mail! True 1 achievement achievement 28 DNK f married n
3 84125 206 3m We watched Rocky Horror Picture Show for the f... We watched Rocky Horror Picture Show for the f... True 1 leisure leisure 28 DNK f married n
4 85125 206 3m We played Mario Kart while drunk. We played Mario Kart while drunk. True 1 leisure leisure 28 DNK f married n
In [32]:
cleaned_hm_adult_cleaned_df = cleaned_hm_df_merge[pd.to_numeric(cleaned_hm_df_merge['age'],errors='coerce').notnull()]
cleaned_hm_adult_cleaned_df['age'] = cleaned_hm_adult_cleaned_df['age'].astype(str).astype(float).astype(int)
cleaned_hm_adult_cleaned_df = cleaned_hm_adult_cleaned_df[cleaned_hm_adult_cleaned_df['age']>30]

/var/folders/x1/cmq6ht1x7ygffd5zpgqj2kv80000gn/T/ipykernel_71323/4083055248.py:2: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  cleaned_hm_adult_cleaned_df['age'] = cleaned_hm_adult_cleaned_df['age'].astype(str).astype(float).astype(int)
In [33]:
cleaned_hm_adult_cleaned_df.head()
Out[33]:
hmid wid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category age country gender marital parenthood
16 27705 5732 24h I picked my daughter up from the airport and w... I picked my daughter up from the airport and w... True 1 bonding affection 64 USA f divorced y
20 27723 389 24h I went shopping I went shopping True 1 leisure leisure 61 USA f married y
21 27923 389 24h My daughter visited me at evening My daughter visited me at evening True 1 affection affection 61 USA f married y
22 45297 389 24h I meet my son and ate a lunch with him I meet my son and ate a lunch with him True 1 affection affection 61 USA f married y
23 83193 389 3m I meet my daughter special one I meet my daughter special one True 1 affection affection 61 USA f married y
In [34]:
adult_married_df = cleaned_hm_adult_cleaned_df[cleaned_hm_adult_cleaned_df['marital']=="married"]
In [35]:
adult_married_df.head()
Out[35]:
hmid wid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category age country gender marital parenthood
20 27723 389 24h I went shopping I went shopping True 1 leisure leisure 61 USA f married y
21 27923 389 24h My daughter visited me at evening My daughter visited me at evening True 1 affection affection 61 USA f married y
22 45297 389 24h I meet my son and ate a lunch with him I meet my son and ate a lunch with him True 1 affection affection 61 USA f married y
23 83193 389 3m I meet my daughter special one I meet my daughter special one True 1 affection affection 61 USA f married y
24 89903 389 3m I meet my son girlfriend I meet my son girlfriend True 1 affection affection 61 USA f married y
In [36]:
adult_married_df.groupby(['ground_truth_category']).agg(['count'])
Out[36]:
hmid wid reflection_period original_hm cleaned_hm modified num_sentence predicted_category age country gender marital parenthood
count count count count count count count count count count count count count
ground_truth_category
achievement 1028 1028 1028 1028 1028 1028 1028 1028 1028 1027 1027 1028 1028
affection 1706 1706 1706 1706 1706 1706 1706 1706 1706 1704 1704 1706 1704
bonding 342 342 342 342 342 342 342 342 342 342 342 342 342
enjoy_the_moment 323 323 323 323 323 323 323 323 323 322 323 323 323
exercise 37 37 37 37 37 37 37 37 37 36 37 37 37
leisure 227 227 227 227 227 227 227 227 227 227 227 227 227
nature 80 80 80 80 80 80 80 80 80 80 80 80 80
In [37]:
marital_recs = []
sns.set()
for martial_state in cleaned_hm_adult_cleaned_df['marital'].unique():
    rec = {
        "name":martial_state,
        "count":len(cleaned_hm_adult_cleaned_df[cleaned_hm_adult_cleaned_df['marital'] == martial_state])
    }
    marital_recs.append(rec)
marital_state_df = pd.DataFrame(marital_recs)
sns.barplot(data=marital_state_df,x="name",y="count")
Out[37]:
<Axes: xlabel='name', ylabel='count'>
In [38]:
recs_parenthood = []
sns.set()
for parenthood_state in cleaned_hm_adult_cleaned_df['parenthood'].unique():
    rec = {
        "name":parenthood_state,
        "count":len(cleaned_hm_adult_cleaned_df[cleaned_hm_adult_cleaned_df['parenthood'] == parenthood_state])
    }
    recs_parenthood.append(rec)
parenthood_df = pd.DataFrame(recs_parenthood)
sns.barplot(data=parenthood_df,x="name",y="count")
Out[38]:
<Axes: xlabel='name', ylabel='count'>
In [39]:
cleaned_hm_adult_married_cleaned_df = cleaned_hm_adult_cleaned_df[cleaned_hm_adult_cleaned_df['marital']=='married']
In [40]:
cleaned_hm_adult_married_cleaned_df.head()
Out[40]:
hmid wid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category age country gender marital parenthood
20 27723 389 24h I went shopping I went shopping True 1 leisure leisure 61 USA f married y
21 27923 389 24h My daughter visited me at evening My daughter visited me at evening True 1 affection affection 61 USA f married y
22 45297 389 24h I meet my son and ate a lunch with him I meet my son and ate a lunch with him True 1 affection affection 61 USA f married y
23 83193 389 3m I meet my daughter special one I meet my daughter special one True 1 affection affection 61 USA f married y
24 89903 389 3m I meet my son girlfriend I meet my son girlfriend True 1 affection affection 61 USA f married y
In [41]:
recs_parenthood = []
sns.set()
for parenthood_state in cleaned_hm_adult_married_cleaned_df['parenthood'].unique():
    rec = {
        "name":parenthood_state,
        "count":len(cleaned_hm_adult_married_cleaned_df[cleaned_hm_adult_married_cleaned_df['parenthood'] == parenthood_state])
    }
    recs_parenthood.append(rec)
parenthood_df = pd.DataFrame(recs_parenthood)
sns.barplot(data=parenthood_df,x="name",y="count")
Out[41]:
<Axes: xlabel='name', ylabel='count'>
In [42]:
cleaned_hm_adult_married_cleaned_df.head()
Out[42]:
hmid wid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category age country gender marital parenthood
20 27723 389 24h I went shopping I went shopping True 1 leisure leisure 61 USA f married y
21 27923 389 24h My daughter visited me at evening My daughter visited me at evening True 1 affection affection 61 USA f married y
22 45297 389 24h I meet my son and ate a lunch with him I meet my son and ate a lunch with him True 1 affection affection 61 USA f married y
23 83193 389 3m I meet my daughter special one I meet my daughter special one True 1 affection affection 61 USA f married y
24 89903 389 3m I meet my son girlfriend I meet my son girlfriend True 1 affection affection 61 USA f married y

Whether the Marital would improve happiness?¶

In [43]:
import pandas as pd

import matplotlib.pyplot as plt
import seaborn as sns
In [44]:
cleaned_hm_csv_file = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/data/cleaned_hm.csv"
cleaned_hm_df = pd.read_csv(cleaned_hm_csv_file)

demographic_csv_file = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/data/demographic.csv" 
demographic_df = pd.read_csv(demographic_csv_file)
In [45]:
cleaned_hm_df = cleaned_hm_df.dropna(subset=['ground_truth_category'])
In [46]:
len(demographic_df)
Out[46]:
10844
In [47]:
len(cleaned_hm_df)
Out[47]:
14125
In [48]:
cleaned_hm_df.head()
Out[48]:
hmid wid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category
3 27676 206 24h We had a serious talk with some friends of our... We had a serious talk with some friends of our... True 2 bonding bonding
5 27678 45 24h I meditated last night. I meditated last night. True 1 leisure leisure
24 27697 498 24h My grandmother start to walk from the bed afte... My grandmother start to walk from the bed afte... True 1 affection affection
32 27705 5732 24h I picked my daughter up from the airport and w... I picked my daughter up from the airport and w... True 1 bonding affection
42 27715 2272 24h when i received flowers from my best friend when i received flowers from my best friend True 1 bonding bonding
In [49]:
cleaned_hm_df_merge = cleaned_hm_df.merge(demographic_df,on='wid')
In [50]:
len(cleaned_hm_df_merge)
Out[50]:
14125
In [51]:
cleaned_hm_df_merge.head()
Out[51]:
hmid wid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category age country gender marital parenthood
0 27676 206 24h We had a serious talk with some friends of our... We had a serious talk with some friends of our... True 2 bonding bonding 28 DNK f married n
1 49534 206 24h I bought my wedding shoes! I bought my wedding shoes! True 1 achievement achievement 28 DNK f married n
2 52451 206 24h My new watch arrived in the mail! My new watch arrived in the mail! True 1 achievement achievement 28 DNK f married n
3 84125 206 3m We watched Rocky Horror Picture Show for the f... We watched Rocky Horror Picture Show for the f... True 1 leisure leisure 28 DNK f married n
4 85125 206 3m We played Mario Kart while drunk. We played Mario Kart while drunk. True 1 leisure leisure 28 DNK f married n
In [52]:
cleaned_hm_df_merge['marital'].unique()
Out[52]:
array(['married', 'single', 'divorced', 'separated', 'widowed', nan],
      dtype=object)
In [53]:
cleaned_hm_df_merge['marital'].isna().sum()
Out[53]:
22
In [59]:
# Initialize an empty list to store records of counts for each marital state
marital_recs = []

# Set the aesthetic style of the plots
sns.set()

# Loop over each unique marital state in the merged dataframe containing cleaned happiness messages
for marital_state in cleaned_hm_df_merge['marital'].unique():
    # Count the number of occurrences for the current marital state
    count = cleaned_hm_df_merge[cleaned_hm_df_merge['marital'] == marital_state].shape[0]
    # Create a dictionary record with the marital state name and its count
    rec = {"name": marital_state, "count": count}
    # Append the record to the list of marital records
    marital_recs.append(rec)

# Convert the list of records into a pandas DataFrame
marital_state_df = pd.DataFrame(marital_recs)

# Use Seaborn to create a bar plot visualizing the frequency of each marital state
sns.barplot(data=marital_state_df, x="name", y="count")
Out[59]:
<Axes: xlabel='name', ylabel='count'>
In [61]:
marital_state_df
Out[61]:
name count
0 married 6059
1 single 7431
2 divorced 471
3 separated 84
4 widowed 58
5 NaN 0
In [63]:
marital_state_df['count'].sum()
Out[63]:
14103
In [64]:
cleaned_hm_df_merge['parenthood'].unique()
Out[64]:
array(['n', 'y', nan], dtype=object)
In [65]:
recs_parenthood = []
sns.set()
for parenthood_state in cleaned_hm_df_merge['parenthood'].unique():
    rec = {
        "name":parenthood_state,
        "count":len(cleaned_hm_df_merge[cleaned_hm_df_merge['parenthood'] == parenthood_state])
    }
    recs_parenthood.append(rec)
parenthood_df = pd.DataFrame(recs_parenthood)
sns.barplot(data=parenthood_df,x="name",y="count")
Out[65]:
<Axes: xlabel='name', ylabel='count'>
In [68]:
cleaned_hm_adult_df.head()
Out[68]:
hmid wid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category age country gender marital parenthood
0 27676 206 24h We had a serious talk with some friends of our... We had a serious talk with some friends of our... True 2 bonding bonding 28 DNK f married n
1 49534 206 24h I bought my wedding shoes! I bought my wedding shoes! True 1 achievement achievement 28 DNK f married n
2 52451 206 24h My new watch arrived in the mail! My new watch arrived in the mail! True 1 achievement achievement 28 DNK f married n
3 84125 206 3m We watched Rocky Horror Picture Show for the f... We watched Rocky Horror Picture Show for the f... True 1 leisure leisure 28 DNK f married n
4 85125 206 3m We played Mario Kart while drunk. We played Mario Kart while drunk. True 1 leisure leisure 28 DNK f married n
In [70]:
cleaned_hm_adult_df = cleaned_hm_df_merge.dropna(subset=['age'])
# This will print any value that cannot be converted to a float
for value in cleaned_hm_df_merge['age']:
    try:
        float(value)
    except ValueError:
        print(value)

prefer not to say
prefer not to say
prefer not to say
prefer not to say
prefer not to say
prefer not to say
prefer not to say
prefer not to say
prefer not to say
prefer not to say
prefer not to say
prefer not to say
čá
In [73]:
# Replace non-numeric values with NaN
cleaned_hm_df_merge['age'] = pd.to_numeric(cleaned_hm_df_merge['age'], errors='coerce')

# Now you can drop NaN values if that's the chosen approach
cleaned_hm_df_merge = cleaned_hm_df_merge.dropna(subset=['age'])

# Perform the filtering
cleaned_hm_adult_df = cleaned_hm_df_merge[cleaned_hm_df_merge['age'] > 30]
In [74]:
cleaned_hm_adult_cleaned_df = cleaned_hm_adult_df[pd.to_numeric(cleaned_hm_adult_df['age'],errors='coerce').notnull()]
In [75]:
cleaned_hm_adult_cleaned_df['age'] = cleaned_hm_adult_cleaned_df['age'].astype(str).astype(float).astype(int)
In [76]:
cleaned_hm_adult_cleaned_df.head()
Out[76]:
hmid wid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category age country gender marital parenthood
16 27705 5732 24h I picked my daughter up from the airport and w... I picked my daughter up from the airport and w... True 1 bonding affection 64 USA f divorced y
20 27723 389 24h I went shopping I went shopping True 1 leisure leisure 61 USA f married y
21 27923 389 24h My daughter visited me at evening My daughter visited me at evening True 1 affection affection 61 USA f married y
22 45297 389 24h I meet my son and ate a lunch with him I meet my son and ate a lunch with him True 1 affection affection 61 USA f married y
23 83193 389 3m I meet my daughter special one I meet my daughter special one True 1 affection affection 61 USA f married y
In [77]:
cleaned_hm_adult_cleaned_df[cleaned_hm_adult_cleaned_df['age']>30]
Out[77]:
hmid wid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category age country gender marital parenthood
16 27705 5732 24h I picked my daughter up from the airport and w... I picked my daughter up from the airport and w... True 1 bonding affection 64 USA f divorced y
20 27723 389 24h I went shopping I went shopping True 1 leisure leisure 61 USA f married y
21 27923 389 24h My daughter visited me at evening My daughter visited me at evening True 1 affection affection 61 USA f married y
22 45297 389 24h I meet my son and ate a lunch with him I meet my son and ate a lunch with him True 1 affection affection 61 USA f married y
23 83193 389 3m I meet my daughter special one I meet my daughter special one True 1 affection affection 61 USA f married y
... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
14116 128565 5216 24h I saw a romantic comedy. I saw a romantic comedy. True 1 leisure leisure 40 USA m single y
14118 128597 5826 24h I found a little extra money in one of my onli... I found a little extra money in one of my onli... True 1 achievement achievement 44 USA m single n
14119 128609 6437 24h After diner I had a nice family walk with my c... After diner I had a nice family walk with my c... True 1 affection affection 31 LVA m married y
14121 128631 8937 24h My best out of town friend called to announce ... My best out of town friend called to announce ... True 1 bonding bonding 61 USA m single n
14123 128758 4428 24h Yesterday my relations came to my house. That ... Yesterday my relations came to my house. That ... True 2 affection affection 34 IND f married y

6237 rows × 14 columns

In [78]:
len(cleaned_hm_adult_cleaned_df)
Out[78]:
6237
In [79]:
cleaned_hm_adult_cleaned_df = cleaned_hm_adult_cleaned_df[cleaned_hm_adult_cleaned_df['age']>30]
In [80]:
len(cleaned_hm_adult_cleaned_df)
Out[80]:
6237

Classification by country¶

In [81]:
marital_recs = []
sns.set()
for martial_state in cleaned_hm_adult_cleaned_df['marital'].unique():
    rec = {
        "name":martial_state,
        "count":len(cleaned_hm_adult_cleaned_df[cleaned_hm_adult_cleaned_df['marital'] == martial_state])
    }
    marital_recs.append(rec)
marital_state_df = pd.DataFrame(marital_recs)
sns.barplot(data=marital_state_df,x="name",y="count")
Out[81]:
<Axes: xlabel='name', ylabel='count'>
In [82]:
recs_parenthood = []
sns.set()
for parenthood_state in cleaned_hm_adult_cleaned_df['parenthood'].unique():
    rec = {
        "name":parenthood_state,
        "count":len(cleaned_hm_adult_cleaned_df[cleaned_hm_adult_cleaned_df['parenthood'] == parenthood_state])
    }
    recs_parenthood.append(rec)
parenthood_df = pd.DataFrame(recs_parenthood)
sns.barplot(data=parenthood_df,x="name",y="count")
Out[82]:
<Axes: xlabel='name', ylabel='count'>
In [83]:
cleaned_hm_adult_married_cleaned_df = cleaned_hm_adult_cleaned_df[cleaned_hm_adult_cleaned_df['marital']=='married']
In [84]:
cleaned_hm_adult_married_cleaned_df.head()
Out[84]:
hmid wid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category age country gender marital parenthood
20 27723 389 24h I went shopping I went shopping True 1 leisure leisure 61 USA f married y
21 27923 389 24h My daughter visited me at evening My daughter visited me at evening True 1 affection affection 61 USA f married y
22 45297 389 24h I meet my son and ate a lunch with him I meet my son and ate a lunch with him True 1 affection affection 61 USA f married y
23 83193 389 3m I meet my daughter special one I meet my daughter special one True 1 affection affection 61 USA f married y
24 89903 389 3m I meet my son girlfriend I meet my son girlfriend True 1 affection affection 61 USA f married y
In [85]:
recs_parenthood = []
sns.set()
for parenthood_state in cleaned_hm_adult_married_cleaned_df['parenthood'].unique():
    rec = {
        "name":parenthood_state,
        "count":len(cleaned_hm_adult_married_cleaned_df[cleaned_hm_adult_married_cleaned_df['parenthood'] == parenthood_state])
    }
    recs_parenthood.append(rec)
parenthood_df = pd.DataFrame(recs_parenthood)
sns.barplot(data=parenthood_df,x="name",y="count")
Out[85]:
<Axes: xlabel='name', ylabel='count'>
In [86]:
cleaned_hm_adult_married_cleaned_df.head()
Out[86]:
hmid wid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category age country gender marital parenthood
20 27723 389 24h I went shopping I went shopping True 1 leisure leisure 61 USA f married y
21 27923 389 24h My daughter visited me at evening My daughter visited me at evening True 1 affection affection 61 USA f married y
22 45297 389 24h I meet my son and ate a lunch with him I meet my son and ate a lunch with him True 1 affection affection 61 USA f married y
23 83193 389 3m I meet my daughter special one I meet my daughter special one True 1 affection affection 61 USA f married y
24 89903 389 3m I meet my son girlfriend I meet my son girlfriend True 1 affection affection 61 USA f married y

Other Relative Exploration¶

In [88]:
import torch
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

import numpy as np
In [91]:
chm_csv_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/data/cleaned_hm.csv"
demographic_csv_path = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/data/demographic.csv"
In [92]:
chm_df = pd.read_csv(chm_csv_path)
chm_df.head()
Out[92]:
hmid wid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category
0 27673 2053 24h I went on a successful date with someone I fel... I went on a successful date with someone I fel... True 1 NaN affection
1 27674 2 24h I was happy when my son got 90% marks in his e... I was happy when my son got 90% marks in his e... True 1 NaN affection
2 27675 1936 24h I went to the gym this morning and did yoga. I went to the gym this morning and did yoga. True 1 NaN exercise
3 27676 206 24h We had a serious talk with some friends of our... We had a serious talk with some friends of our... True 2 bonding bonding
4 27677 6227 24h I went with grandchildren to butterfly display... I went with grandchildren to butterfly display... True 1 NaN affection
In [93]:
demographic_df = pd.read_csv(demographic_csv_path)
demographic_df.head()
Out[93]:
wid age country gender marital parenthood
0 1 37.0 USA m married y
1 2 29.0 IND m married y
2 3 25 IND m single n
3 4 32 USA m married y
4 5 29 USA m married y
In [94]:
merge_df = demographic_df.merge(chm_df,on='wid')
merge_df.head()
Out[94]:
wid age country gender marital parenthood hmid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category
0 1 37.0 USA m married y 27854 24h My wife and i are celebrating our ten year ann... My wife and i are celebrating our ten year ann... True 1 NaN affection
1 1 37.0 USA m married y 28054 24h My mother called out of the blue to tell me ho... My mother called out of the blue to tell me ho... True 1 affection affection
2 1 37.0 USA m married y 28254 24h Today i took the day off from my part time job... Today i took the day off from my part time job... True 1 NaN affection
3 1 37.0 USA m married y 28489 24h I just got a $21 bonus on a mturk task! I just got a $21 bonus on a mturk task! True 1 NaN achievement
4 1 37.0 USA m married y 28991 24h My wife cooked me a surprise dinner to take to... My wife cooked me a surprise dinner to take to... True 1 NaN affection
In [95]:
merge_df.drop_duplicates(subset=['wid'],inplace=True)
merge_df.head()
Out[95]:
wid age country gender marital parenthood hmid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category
0 1 37.0 USA m married y 27854 24h My wife and i are celebrating our ten year ann... My wife and i are celebrating our ten year ann... True 1 NaN affection
90 2 29.0 IND m married y 27674 24h I was happy when my son got 90% marks in his e... I was happy when my son got 90% marks in his e... True 1 NaN affection
174 3 25 IND m single n 27681 24h YESTERDAY MY MOMS BIRTHDAY SO I ENJOYED YESTERDAY MY MOMS BIRTHDAY SO I ENJOYED True 1 NaN enjoy_the_moment
258 4 32 USA m married y 27714 24h I received an award for my work on a tough loa... I received an award for my work on a tough loa... True 1 NaN achievement
354 5 29 USA m married y 27838 24h I was able to play my video game that I enjoy ... I was able to play my video game that I enjoy ... True 1 leisure leisure
In [96]:
merge_df = merge_df[['wid','age','country','gender','marital','parenthood','ground_truth_category','predicted_category']]
In [97]:
merge_df.head()
Out[97]:
wid age country gender marital parenthood ground_truth_category predicted_category
0 1 37.0 USA m married y NaN affection
90 2 29.0 IND m married y NaN affection
174 3 25 IND m single n NaN enjoy_the_moment
258 4 32 USA m married y NaN achievement
354 5 29 USA m married y leisure leisure
In [98]:
merge_df['country'].unique()
Out[98]:
array(['USA', 'IND', 'VNM', 'THA', 'GBR', 'SRB', 'BGR', 'BRA', 'BGD',
       'CAN', 'PRT', 'PHL', 'VEN', 'MKD', 'DOM', 'DNK', 'IDN', 'AUS',
       'URY', 'JAM', 'EGY', 'DEU', 'KEN', 'MDA', 'IRL', 'NGA', 'ALB',
       'MEX', 'TTO', 'KWT', 'RUS', 'LTU', 'PRI', 'SWE', 'PER', 'PAK',
       'TUR', 'FRA', 'ARE', 'COL', 'GRC', 'ROU', 'ARM', 'MAC', 'SGP',
       'UGA', 'POL', 'NIC', 'JPN', 'ESP', 'TWN', 'FIN', 'ITA', 'GHA',
       'AUT', 'NPL', 'NZL', 'NLD', 'KOR', 'LKA', 'KNA', 'ZAF', 'BEL',
       'ISL', 'DZA', 'AFG', 'GMB', nan, 'ASM', 'MYS', 'CZE', 'MLT', 'SVN',
       'UMI', 'ARG', 'SAU', 'EST', 'ECU', 'ZMB', 'CHL', 'MAR', 'ISR',
       'LVA', 'KAZ', 'NOR', 'CRI', 'BRB', 'HRV', 'ETH', 'TCA', 'BHS',
       'VIR', 'GTM', 'IRQ', 'HKG', 'MUS', 'TUN', 'SUR', 'SLV', 'CYP',
       'UKR'], dtype=object)
In [99]:
country_series = merge_df['country'].value_counts()[:10]
# country_series
sns.set()
barplot  = sns.barplot(x=country_series.index,y=country_series.values)

for p in barplot.patches:
    barplot.annotate(format(p.get_height(), '.1f'), 
                     (p.get_x() + p.get_width() / 2., p.get_height()),
                     ha = 'center', va = 'center', 
                     xytext = (0, 9), 
                     textcoords = 'offset points')
In [100]:
merge_country_df = merge_df[merge_df['country'] == 'USA']
In [101]:
merge_country_df.head()
Out[101]:
wid age country gender marital parenthood ground_truth_category predicted_category
0 1 37.0 USA m married y NaN affection
258 4 32 USA m married y NaN achievement
354 5 29 USA m married y leisure leisure
510 7 34 USA m married y achievement achievement
678 9 61 USA f married y NaN affection
In [102]:
merge_country_df['age'] = pd.to_numeric(merge_country_df['age'], errors='coerce')

/var/folders/x1/cmq6ht1x7ygffd5zpgqj2kv80000gn/T/ipykernel_71323/1739328044.py:1: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  merge_country_df['age'] = pd.to_numeric(merge_country_df['age'], errors='coerce')
In [103]:
merge_country_df['age'] = merge_country_df['age'].fillna(0).astype(int)

/var/folders/x1/cmq6ht1x7ygffd5zpgqj2kv80000gn/T/ipykernel_71323/813599853.py:1: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  merge_country_df['age'] = merge_country_df['age'].fillna(0).astype(int)
In [104]:
marital_series = merge_country_df[merge_country_df['age']>16]['marital'].value_counts()
barplot  = sns.barplot(x=marital_series.index,y=marital_series.values)

for p in barplot.patches:
    barplot.annotate(format(p.get_height(), '.1f'), 
                     (p.get_x() + p.get_width() / 2., p.get_height()),
                     ha = 'center', va = 'center', 
                     xytext = (0, 9), 
                     textcoords = 'offset points')
In [105]:
age_gt_16_df = merge_country_df[merge_country_df['age']>16]
In [106]:
age_gt_16_df.head()
Out[106]:
wid age country gender marital parenthood ground_truth_category predicted_category
0 1 37 USA m married y NaN affection
258 4 32 USA m married y NaN achievement
354 5 29 USA m married y leisure leisure
510 7 34 USA m married y achievement achievement
678 9 61 USA f married y NaN affection
In [107]:
bins = [16, 20, 25, 30,35,40,45,50,55,60]
labels = ['16-20', '21-25', '26-30','31-35', '36-40', '41-45','46-50', '51-55', '56-60']
age_gt_16_df['range'] = pd.cut(age_gt_16_df['age'], bins=bins, labels=labels)

/var/folders/x1/cmq6ht1x7ygffd5zpgqj2kv80000gn/T/ipykernel_71323/2921349568.py:3: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  age_gt_16_df['range'] = pd.cut(age_gt_16_df['age'], bins=bins, labels=labels)
In [108]:
age_gt_16_df.head()
Out[108]:
wid age country gender marital parenthood ground_truth_category predicted_category range
0 1 37 USA m married y NaN affection 36-40
258 4 32 USA m married y NaN achievement 31-35
354 5 29 USA m married y leisure leisure 26-30
510 7 34 USA m married y achievement achievement 31-35
678 9 61 USA f married y NaN affection NaN
In [109]:
range_count = age_gt_16_df['range'].value_counts()
barplot  = sns.barplot(x=range_count.index,y=range_count.values)

for p in barplot.patches:
    barplot.annotate(format(p.get_height(), '.1f'), 
                     (p.get_x() + p.get_width() / 2., p.get_height()),
                     ha = 'center', va = 'center', 
                     xytext = (0, 9), 
                     textcoords = 'offset points')
In [110]:
married_range_count = age_gt_16_df[age_gt_16_df['marital']=='married']['range'].value_counts()
In [111]:
barplot  = sns.barplot(x=married_range_count.index,y=married_range_count.values)

for p in barplot.patches:
    barplot.annotate(format(p.get_height(), '.1f'), 
                     (p.get_x() + p.get_width() / 2., p.get_height()),
                     ha = 'center', va = 'center', 
                     xytext = (0, 9), 
                     textcoords = 'offset points')
In [112]:
sinle_range_count = age_gt_16_df[age_gt_16_df['marital']=='single']['range'].value_counts()
In [113]:
barplot  = sns.barplot(x=sinle_range_count.index,y=sinle_range_count.values)

for p in barplot.patches:
    barplot.annotate(format(p.get_height(), '.1f'), 
                     (p.get_x() + p.get_width() / 2., p.get_height()),
                     ha = 'center', va = 'center', 
                     xytext = (0, 9), 
                     textcoords = 'offset points')
In [114]:
single_df = age_gt_16_df[age_gt_16_df['marital']=='single']
single_df
Out[114]:
wid age country gender marital parenthood ground_truth_category predicted_category range
765 10 27 USA m single n NaN exercise 26-30
936 12 25 USA f single n NaN affection 21-25
1239 16 35 USA m single n NaN affection 31-35
1302 17 30 USA m single n NaN affection 26-30
1380 18 30 USA m single n NaN enjoy_the_moment 26-30
... ... ... ... ... ... ... ... ... ...
100517 13834 22 USA m single n NaN affection 21-25
100520 13835 25 USA m single n NaN bonding 21-25
100523 13836 31 USA m single y NaN achievement 31-35
100526 13837 22 USA f single n enjoy_the_moment enjoy_the_moment 21-25
100532 13839 24 USA f single y NaN affection 21-25

4916 rows × 9 columns

In [115]:
single_series = single_df['predicted_category'].value_counts()
barplot  = sns.barplot(x=single_series.index,y=single_series.values)
barplot.set_xticklabels(barplot.get_xticklabels(), rotation=90)
for p in barplot.patches:
    barplot.annotate(format(p.get_height(), '.1f'), 
                     (p.get_x() + p.get_width() / 2., p.get_height()),
                     ha = 'center', va = 'center', 
                     xytext = (0, 9), 
                     textcoords = 'offset points')
In [116]:
married_df = age_gt_16_df[age_gt_16_df['marital']=='married']
married_series = married_df['predicted_category'].value_counts()
barplot  = sns.barplot(x=married_series.index,y=married_series.values)
barplot.set_xticklabels(barplot.get_xticklabels(), rotation=90)
for p in barplot.patches:
    barplot.annotate(format(p.get_height(), '.1f'), 
                     (p.get_x() + p.get_width() / 2., p.get_height()),
                     ha = 'center', va = 'center', 
                     xytext = (0, 9), 
                     textcoords = 'offset points')
In [117]:
sns.set_theme(style="whitegrid")
married_df['value'] = 1
married_df.head()

/var/folders/x1/cmq6ht1x7ygffd5zpgqj2kv80000gn/T/ipykernel_71323/1074539863.py:2: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  married_df['value'] = 1
Out[117]:
wid age country gender marital parenthood ground_truth_category predicted_category range value
0 1 37 USA m married y NaN affection 36-40 1
258 4 32 USA m married y NaN achievement 31-35 1
354 5 29 USA m married y leisure leisure 26-30 1
510 7 34 USA m married y achievement achievement 31-35 1
678 9 61 USA f married y NaN affection NaN 1
In [123]:
category_grouped_df = married_df.groupby(['gender','predicted_category']).sum().reset_index()
g = sns.catplot(
    x='predicted_category', 
    y='value', 
    hue='gender', 
    data=category_grouped_df, 
    kind='bar',
     palette="dark", alpha=.6, height=6,
)

g.despine(left=True)
g.set_axis_labels("predicted_category", "count")
for ax in g.axes.flat:
    for label in ax.get_xticklabels():
        label.set_rotation(90)

/var/folders/x1/cmq6ht1x7ygffd5zpgqj2kv80000gn/T/ipykernel_71323/283923920.py:1: FutureWarning: The default value of numeric_only in DataFrameGroupBy.sum is deprecated. In a future version, numeric_only will default to False. Either specify numeric_only or select only columns which should be valid for the function.
  category_grouped_df = married_df.groupby(['gender','predicted_category']).sum().reset_index()
In [124]:
married_df
Out[124]:
wid age country gender marital parenthood ground_truth_category predicted_category range value
0 1 37 USA m married y NaN affection 36-40 1
258 4 32 USA m married y NaN achievement 31-35 1
354 5 29 USA m married y leisure leisure 26-30 1
510 7 34 USA m married y achievement achievement 31-35 1
678 9 61 USA f married y NaN affection NaN 1
... ... ... ... ... ... ... ... ... ... ...
100493 13822 32 USA f married n NaN affection 31-35 1
100502 13828 29 USA m married y NaN affection 26-30 1
100511 13832 34 USA f married y NaN affection 31-35 1
100514 13833 31 USA m married y affection affection 31-35 1
100529 13838 38 USA f married y NaN enjoy_the_moment 36-40 1

3613 rows × 10 columns

In [125]:
single_df['value'] = 1
single_df

/var/folders/x1/cmq6ht1x7ygffd5zpgqj2kv80000gn/T/ipykernel_71323/3356902141.py:1: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  single_df['value'] = 1
Out[125]:
wid age country gender marital parenthood ground_truth_category predicted_category range value
765 10 27 USA m single n NaN exercise 26-30 1
936 12 25 USA f single n NaN affection 21-25 1
1239 16 35 USA m single n NaN affection 31-35 1
1302 17 30 USA m single n NaN affection 26-30 1
1380 18 30 USA m single n NaN enjoy_the_moment 26-30 1
... ... ... ... ... ... ... ... ... ... ...
100517 13834 22 USA m single n NaN affection 21-25 1
100520 13835 25 USA m single n NaN bonding 21-25 1
100523 13836 31 USA m single y NaN achievement 31-35 1
100526 13837 22 USA f single n enjoy_the_moment enjoy_the_moment 21-25 1
100532 13839 24 USA f single y NaN affection 21-25 1

4916 rows × 10 columns

In [126]:
single_df_grp =single_df.groupby(['predicted_category'])['value'].sum()
single_df_grp
Out[126]:
predicted_category
achievement         1902
affection           1639
bonding              575
enjoy_the_moment     417
exercise              58
leisure              244
nature                81
Name: value, dtype: int64
In [127]:
married_df_grp = married_df.groupby(['predicted_category'])['value'].sum()
married_df_grp.values
Out[127]:
array([1009, 2008,  199,  203,   31,   98,   65])

3. How to improve happniess among the marital and singel?¶

Analysis of the Sources of Marital Happiness¶

In [128]:
import pandas as pd

import matplotlib.pyplot as plt
import seaborn as sns
In [130]:
cleaned_hm_csv_file = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/data/cleaned_hm.csv"
cleaned_hm_df = pd.read_csv(cleaned_hm_csv_file)

demographic_csv_file = "/Users/chenjianfeng/Github/new_Colin/ads-spring2024-project1-Colin-chan1366/data/demographic.csv" 
demographic_df = pd.read_csv(demographic_csv_file)
In [131]:
cleaned_hm_df.head()
Out[131]:
hmid wid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category
0 27673 2053 24h I went on a successful date with someone I fel... I went on a successful date with someone I fel... True 1 NaN affection
1 27674 2 24h I was happy when my son got 90% marks in his e... I was happy when my son got 90% marks in his e... True 1 NaN affection
2 27675 1936 24h I went to the gym this morning and did yoga. I went to the gym this morning and did yoga. True 1 NaN exercise
3 27676 206 24h We had a serious talk with some friends of our... We had a serious talk with some friends of our... True 2 bonding bonding
4 27677 6227 24h I went with grandchildren to butterfly display... I went with grandchildren to butterfly display... True 1 NaN affection
In [132]:
cleaned_hm_df = cleaned_hm_df.dropna(subset=['ground_truth_category'])
In [133]:
merged_df = demographic_df.merge(cleaned_hm_df,on='wid')
merged_df.head()
Out[133]:
wid age country gender marital parenthood hmid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category
0 1 37.0 USA m married y 28054 24h My mother called out of the blue to tell me ho... My mother called out of the blue to tell me ho... True 1 affection affection
1 1 37.0 USA m married y 57637 24h I got to take the day off from my part time jo... I got to take the day off from my part time jo... True 1 affection affection
2 1 37.0 USA m married y 64498 3m my son had a great time on his 8th birthday. my son had a great time on his 8th birthday. True 1 affection affection
3 1 37.0 USA m married y 80306 3m I got to have a three day weekend which i spen... I got to have a three day weekend which i spen... True 1 affection affection
4 1 37.0 USA m married y 88307 3m My fiance proposed to me in front of my family... My fiance proposed to me in front of my family... True 1 affection affection
In [134]:
merged_df = merged_df[merged_df['country']=='USA']
In [135]:
merged_df['age'] = pd.to_numeric(merged_df['age'], errors='coerce')
merged_df['age'] = merged_df['age'].fillna(0).astype(int)
adult_df = merged_df[merged_df['age']>16]
In [136]:
married_df = adult_df[adult_df['marital'] == 'married']
In [137]:
married_df.head()
Out[137]:
wid age country gender marital parenthood hmid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category
0 1 37 USA m married y 28054 24h My mother called out of the blue to tell me ho... My mother called out of the blue to tell me ho... True 1 affection affection
1 1 37 USA m married y 57637 24h I got to take the day off from my part time jo... I got to take the day off from my part time jo... True 1 affection affection
2 1 37 USA m married y 64498 3m my son had a great time on his 8th birthday. my son had a great time on his 8th birthday. True 1 affection affection
3 1 37 USA m married y 80306 3m I got to have a three day weekend which i spen... I got to have a three day weekend which i spen... True 1 affection affection
4 1 37 USA m married y 88307 3m My fiance proposed to me in front of my family... My fiance proposed to me in front of my family... True 1 affection affection
In [138]:
married_df['value'] = 1

/var/folders/x1/cmq6ht1x7ygffd5zpgqj2kv80000gn/T/ipykernel_71323/430985916.py:1: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  married_df['value'] = 1
In [139]:
married_df.head()
Out[139]:
wid age country gender marital parenthood hmid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category value
0 1 37 USA m married y 28054 24h My mother called out of the blue to tell me ho... My mother called out of the blue to tell me ho... True 1 affection affection 1
1 1 37 USA m married y 57637 24h I got to take the day off from my part time jo... I got to take the day off from my part time jo... True 1 affection affection 1
2 1 37 USA m married y 64498 3m my son had a great time on his 8th birthday. my son had a great time on his 8th birthday. True 1 affection affection 1
3 1 37 USA m married y 80306 3m I got to have a three day weekend which i spen... I got to have a three day weekend which i spen... True 1 affection affection 1
4 1 37 USA m married y 88307 3m My fiance proposed to me in front of my family... My fiance proposed to me in front of my family... True 1 affection affection 1
In [140]:
sns.set_theme(style="whitegrid",palette="pastel")
married_series = married_df.groupby(['ground_truth_category'])['value'].sum()
barplot  = sns.barplot(x=married_series.index,y=married_series.values)
barplot.set_xticklabels(barplot.get_xticklabels(), rotation=30)
for p in barplot.patches:
    barplot.annotate(format(p.get_height(), '.1f'), 
                     (p.get_x() + p.get_width() / 2., p.get_height()),
                     ha = 'center', va = 'center', 
                     xytext = (0, 9), 
                     textcoords = 'offset points')
In [141]:
married_df[married_df['ground_truth_category']=='affection'].to_csv("married_affection.csv")
In [142]:
single_df = adult_df[adult_df['marital'] == 'single']
In [145]:
single_df.head(2)
Out[145]:
wid age country gender marital parenthood hmid reflection_period original_hm cleaned_hm modified num_sentence ground_truth_category predicted_category
125 10 27 USA m single n 40557 24h I went to see The Boss Baby with my little bro... I went to see The Boss Baby with my little bro... True 2 affection affection
126 10 27 USA m single n 46740 24h I played my favorite game with one of my favor... I played my favorite game with one of my favor... True 2 bonding bonding
In [148]:
# single_series = single_df.groupby(['ground_truth_category'])['value'].sum()
# barplot  = sns.barplot(x=single_series.index,y=single_series.values)
# barplot.set_xticklabels(barplot.get_xticklabels(), rotation=30)
# for p in barplot.patches:
#     barplot.annotate(format(p.get_height(), '.1f'), 
#                      (p.get_x() + p.get_width() / 2., p.get_height()),
#                      ha = 'center', va = 'center', 
#                      xytext = (0, 9), 
#                      textcoords = 'offset points')
In [ ]:
# single_df[single_df['ground_truth_category']=='achievement'].to_csv("single_achievement.csv")